adibender
commented
5 months ago @LingfengLuo0510 thanks for the updates. Here are some additional notes. There are still some points I haven't checked, because of the issues below, mostly concerning the paper and documentation. Those concerned points are:
- reproducibility -> see comment regarding real data example
- performance (claims) -> see comments below
- state of the filed -> not really describe what alternative approaches exist
- references -> related to point above
The main improvement for the paper I think would be to make the the scope and potential limitations of the package more clear:
- you mention \beta(t) in the paper, but I think the types of models that can be fit should be made more clear, i.e. varying coefficient models of the form f(t)x, i.e. (potentially) non-linear in time, but linear in x, in contrast for example to f(x)t or f(t,x).
- Implementation wise, users can not choose which type of effect to fit, i.e. one can only specify covariates and they will automatically be assumed to have f(t)x, but one cannot chose some x to be beta x or interactions between different Xs, non-linear effects f(x), etc., which is fine, but should be briefly made explicit
- Line 44: "compared to existing computational packages..." Here are some performance claims, but the paper does not explicitly mention which packages were compared and I couldn't find the comparisons also in the referenced papers (see also my attached R code where I compare surtvep to 2 approaches, my own PAMMs and Aalens additive hazards model from timereg package, where for the simple case they appear to be faster)
- It's also not clear from the paper in which situations the package applies, n>p or n<<p?
- The Data Example is not reproducible as it only links to the Homepage, but not clear which data exactly was downloaded, how it was preprocessed, etc.
- It's not clear for which survival tasks it is applicable, only to right-censored data or also left-/interval censored, competing risks, multi-state models, etc.
To be clear, I don't ask to run some addtional models or change the implementation, but scope, limitations, overview of existing methods and performance claims should be adapted/verified in the paper. @osorensen has to decide regarding the reproducibility aspect of the real data example.
Some minor comments
- L59: "Breslow ... number of times" probably should be "number of ties"?
- L95: mentions some hypothesis testing capabilities, etc. (add references to the papers where these have been established)
- 104: Link in 104 leads to main page, therein is a section to "Detailed tutorial" but link goes back to landing page
- Figure 3 not reproducible: https://github.com/UM-KevinHe/surtvep/blob/JOSS/JOSS/GenerateRealDataPlot.R reads data from non-accessible location.
- for the figure here: https://um-kevinhe.github.io/surtvep/articles/surtvep.html#quick-start better to use
coord_cartesian(ylim = c(y1,y2))rather thanylim(c(y1,y2))
Edit
Here is the code I mentioned: Note, the Aalen model produces cumulative (time-varying) coefficients, but these are equivalent to f(t)*x type hazards models as shown in Figure 5 here: https://arxiv.org/pdf/1806.01042.pdf?
# This is the example from the paper
library(surtvep)
library(ggplot2)
library(timereg)
library(pammtools)
data("ExampleData")
# data prep
z <- ExampleData$z
time <- ExampleData$time
event <- ExampleData$event
# non-penalized model
fit.tv <- coxtv(z = z, event = event, time = time)
# penalized model
fit.penalize <- coxtp(z = z, event = event, time = time)
fit.ic <- IC(fit.penalize)
# Visualization
plot(fit.tv, ylim = c(-3,10), parm = "X1") + geom_hline(yintercept = 1, col = 2)
plot(fit.ic$model.mAIC, ylim = c(-3,10), par = "X2")
# illustration with PAMMs
# data prep
df <- data.frame(time = time, event = event)
df <- cbind.data.frame(df, z)
colnames(df) <- c("time", "event", "z1", "z2")
# data trafo
ped <- as_ped(Surv(time, event) ~ ., cut = seq(0, 3, length.out = 100), data = df)
# model fit
pam <- pamm(ped_status ~ s(tend) + s(tend, by = z1) + s(tend, by = z2),
data = ped, engine = "bam", method = "fREML", discrete = TRUE)
summary(pam, 1)
# recovers true effects
layout(matrix(1:2, nrow = 1))
plot(pam, select = 2)
abline(h = 1, lty = 2, col = 2)
plot(pam, select = 3)
curve(sin(3 *pi * x / 4), 0, 3, add = TRUE, col = 2, lty = 2)
# Runtime comparisons
# -surtvep
# - pamm
# - aalen
t_surtvep <- system.time({
fit.penalize <- coxtp(z = z, event = event, time = time)
fit.ic <- IC(fit.penalize)
})
t_pam_large <- system.time({
ped <- as_ped(Surv(time, event) ~ ., data = df)
pam <- pamm(
ped_status ~ s(tend) + s(tend, by = z1) + s(tend, by = z2),
data = ped, engine = "bam", method = "fREML", discrete = TRUE)
})
t_pam_small <- system.time({
ped <- as_ped(Surv(time, event) ~ ., cut = seq(0, 3, length.out = 100), data = df)
pam <- pamm(
ped_status ~ s(tend) + s(tend, by = z1) + s(tend, by = z2),
data = ped, engine = "bam", method = "fREML", discrete = TRUE)
})
t_aalen <- system.time({
aal <- aalen(Surv(time, event) ~ z1 + z2, data = df, n.sim = 100)
})
rbind(t_surtvep, t_pam_large, t_pam_small, t_aalen)
t_pam_large[3]/t_surtvep[3]
t_surtvep[3]/t_pam_small[3]
osorensen
editorialbot
LingfengLuo0510
turgeonmaxime
crvernon
Submitting author: !--author-handle-->@LingfengLuo0510<!--end-author-handle-- (Lingfeng Luo) Repository: https://github.com/UM-KevinHe/surtvep Branch with paper.md (empty if default branch): JOSS Version: v1.0.0 Editor: !--editor-->@osorensen<!--end-editor-- Reviewers: @adibender, @turgeonmaxime Archive: 10.5281/zenodo.12575049
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@adibender & @turgeonmaxime, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @osorensen know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @turgeonmaxime
📝 Checklist for @adibender