Skip to contents

AUC Test for Paired Two-sample Measurements

Source: R/pairedroc.R
aucTest.Rd

[Experimental]

This function compares two AUC of paired two-sample diagnostic assays by standardized difference method, which has a little difference in SE calculation with unpaired design. In order to compare the two assays, this function provides three assessments including 'difference', 'non-inferiority' and 'superiority'. This method of comparing is referred from Liu(2006)'s article that can be found in reference section below.

Usage

aucTest(
  x,
  y,
  response,
  h0 = 0,
  conf.level = 0.95,
  method = c("difference", "non-inferiority", "superiority"),
  ...
)

Arguments

x

(numeric)
reference/standard diagnostic assay.

y

(numeric)
test diagnostic assay.

response

(numeric or factor)
a vector of responses to represent the type of classes, typically encoded with 0(controls) and 1(cases).

h0

(numeric)
a specified hypothesized value of the margin between the two assays, default is 0 for difference method. If you select the non-inferiority method, the h0 should be negative value. And if select superiority method, then it's non-negative value.

conf.level

(numeric)
significance level between 0 and 1 (non-inclusive) for the returned confidence interval.

method

(string)
string specifying the type of hypothesis test, must be one of "difference" (default), "non-inferiority" or "superiority".

...

other arguments to be passed to pROC::roc().

Value

A RefInt object contains relevant results in comparing the paired ROC of two-sample assays.

Details

If the samples are not considered independent, such as in a paired design, the SE can not be computed by the method of Delong provided in pROC package. Here the aucTest function use the standardized difference approach from Liu(2006) publication to compute the SE and corresponding hypothesis test statistic for a paired design study.

  • difference is to test the difference between two diagnostic tests, the default h0 is zero.

  • non-inferiority is to test the new diagnostic tests is no worse than the standard diagnostic test in a specific margin, but the same time maybe it's safer, easier to administer or cost less.

  • superiority is to test the test the new diagnostic tests is better than the standard diagnostic test in a specific margin(default is zero), having better efficacy.

Note

The test of significance for the difference is not equal to the result of EP24A2 Appendix D. Table D2. Because the Table D2 uses the method of Hanley & McNeil (1982), whereas this function here uses the method of DeLong et al. (1988), which results in the difference of SE. Thus the corresponding Z statistic and P value will be not equal as well.

References

Jen-Pei Liu (2006) "Tests of equivalence and non-inferiority for diagnostic accuracy based on the paired areas under ROC curves". Statist. Med. , 25:1219–1238. DOI: 10.1002/sim.2358.

Examples

data("ldlroc")
# H0 : Difference between areas = 0:
aucTest(x = ldlroc$LDL, y = ldlroc$OxLDL, response = ldlroc$Diagnosis)
#> Setting levels: control = 0, case = 1
#> Setting direction: controls < cases
#> 
#> The hypothesis for testing difference based on Paired ROC curve
#> 
#>  Test assay:
#>   Area under the curve: 0.7995
#>   Standard Error(SE): 0.0620
#>   95% Confidence Interval(CI): 0.6781-0.9210 (DeLong)
#> 
#>  Reference/standard assay:
#>   Area under the curve: 0.5617
#>   Standard Error(SE): 0.0836
#>   95% Confidence Interval(CI): 0.3979-0.7255 (DeLong)
#> 
#>  Comparison of Paired AUC:
#>   Alternative hypothesis: the difference in AUC is difference to 0
#>   Difference of AUC: 0.2378
#>   Standard Error(SE): 0.0790
#>   95% Confidence Interval(CI): 0.0829-0.3927 (standardized differenec method)
#>   Z: 3.0088
#>   Pvalue: 0.002623

# H0 : Superiority margin <= 0.1:
aucTest(
  x = ldlroc$LDL, y = ldlroc$OxLDL, response = ldlroc$Diagnosis,
  method = "superiority", h0 = 0.1
)
#> Setting levels: control = 0, case = 1
#> Setting direction: controls < cases
#> 
#> The hypothesis for testing superiority based on Paired ROC curve
#> 
#>  Test assay:
#>   Area under the curve: 0.7995
#>   Standard Error(SE): 0.0620
#>   95% Confidence Interval(CI): 0.6781-0.9210 (DeLong)
#> 
#>  Reference/standard assay:
#>   Area under the curve: 0.5617
#>   Standard Error(SE): 0.0836
#>   95% Confidence Interval(CI): 0.3979-0.7255 (DeLong)
#> 
#>  Comparison of Paired AUC:
#>   Alternative hypothesis: the difference in AUC is superiority to 0.1
#>   Difference of AUC: 0.2378
#>   Standard Error(SE): 0.0790
#>   95% Confidence Interval(CI): 0.0829-0.3927 (standardized differenec method)
#>   Z: 1.7436
#>   Pvalue: 0.04061

# H0 : Non-inferiority margin <= -0.1:
aucTest(
  x = ldlroc$LDL, y = ldlroc$OxLDL, response = ldlroc$Diagnosis,
  method = "non-inferiority", h0 = -0.1
)
#> Setting levels: control = 0, case = 1
#> Setting direction: controls < cases
#> 
#> The hypothesis for testing non-inferiority based on Paired ROC curve
#> 
#>  Test assay:
#>   Area under the curve: 0.7995
#>   Standard Error(SE): 0.0620
#>   95% Confidence Interval(CI): 0.6781-0.9210 (DeLong)
#> 
#>  Reference/standard assay:
#>   Area under the curve: 0.5617
#>   Standard Error(SE): 0.0836
#>   95% Confidence Interval(CI): 0.3979-0.7255 (DeLong)
#> 
#>  Comparison of Paired AUC:
#>   Alternative hypothesis: the difference in AUC is non-inferiority to -0.1
#>   Difference of AUC: 0.2378
#>   Standard Error(SE): 0.0790
#>   95% Confidence Interval(CI): 0.0829-0.3927 (standardized differenec method)
#>   Z: 4.2739
#>   Pvalue: 9.606e-06