Summarize a internal_validate object

Usage

# S3 method for class 'internal_validate'
summary(object, ignore_scores = "^cal_plot", ...)

Arguments

object: created by call to validate
ignore_scores: a string used to identify scores to omit from summary. score_binary produces scores with prefix 'cal_plot' when a calibration plot is desired (see cal_plot) and these are ignored by default.
...: ignored

Value

a data.frame with 4 columns (apparent score, optimism, bias-corrected score, number of successful resamples/folds) and one row per score. Not all methods produce an optimism estimate so this row may be all NA. If confidence intervals have been added for all measures via confint.internal_validate, two additional columns containing lower and upper bounds for bias-corrected performance.

Examples

library(pminternal)
set.seed(456)
# simulate data with two predictors that interact
dat <- pmcalibration::sim_dat(N = 2000, a1 = -2, a3 = -.3)
mean(dat$y)
#> [1] 0.1985
dat$LP <- NULL # remove linear predictor

# fit a (misspecified) logistic regression model
m1 <- glm(y ~ ., data=dat, family="binomial")

# internal validation of m1 via bootstrap optimism with 10 resamples
# B = 10 for example but should be >= 200 in practice
m1_iv <- validate(m1, method="boot_optimism", B=10)
#> It is recommended that B >= 200 for bootstrap validation
summary(m1_iv)
#>           Apparent Optimism Corrected  n
#> C           0.7779  0.00158    0.7764 10
#> Brier       0.1335 -0.00111    0.1346 10
#> Intercept   0.0000 -0.01917    0.0192 10
#> Slope       1.0000  0.00083    0.9992 10
#> Eavg        0.0076  0.00516    0.0024 10
#> E50         0.0064  0.00381    0.0026 10
#> E90         0.0115  0.00882    0.0027 10
#> Emax        0.0580  0.07771   -0.0197 10
#> ECI         0.0110  0.03656   -0.0256 10