A MAPE (in)stability plot shows mean absolute predictor error (average absolute difference between original estimated risk and risk from B bootstrap models) as a function of apparent estimated risk (prediction from original/development model). See Riley and Collins (2023).
Arguments
- x
an object produced by
validatewith method = "boot_\*" (orboot_optimismwith method="boot")- xlim
x limits (default = range of estimated risks)
- ylim
y limits (default = c(0, maximum mape))
- xlab
a title for the x axis
- ylab
a title for the y axis
- pch
plotting character (default = 16)
- cex
controls point size (default = 1)
- col
color of points (default = grDevices::grey(.5, .5))
Value
plots calibration (in)stability. Invisibly returns a list containing data for each curve (p=x-axis, pc=y-axis). The first element of this list is the apparent curve (original model on original outcome).
References
Riley RD, Collins GS. (2023). Stability of clinical prediction models developed using statistical or machine learning methods. Biom J. doi:10.1002/bimj.202200302. Epub ahead of print.
Examples
set.seed(456)
# simulate data with two predictors that interact
dat <- pmcalibration::sim_dat(N = 2000, a1 = -2, a3 = -.3)
mean(dat$y)
#> [1] 0.1985
dat$LP <- NULL # remove linear predictor
# fit a (misspecified) logistic regression model
m1 <- glm(y ~ ., data=dat, family="binomial")
# internal validation of m1 via bootstrap optimism with 10 resamples
# B = 10 for example but should be >= 200 in practice
m1_iv <- validate(m1, method="boot_optimism", B=10)
#> It is recommended that B >= 200 for bootstrap validation
mape_stability(m1_iv)
