Closed esther1262 closed 12 months ago
Your pars.fmsm
call is on a different model fit
, and the previous calls worked on a model called fitlog
. Are these different models? If so, the parameter estimates will be different. Likewise if you simulate a different dataset and fit a model to it, then the parameter estimates will be different, because the data are different.
The $coefficients are log-transformed, but the print output is on the natural scale. This is documented in the help page.
There's nothing in flexsurv specifically designed to simulate data from Markov models with flexible time-to-event distributions. But if your transition-specific distributions are all exponential, then the semi-Markov model will also be Markov, so you can use sim.fmsm
Hi Chris, I updated my codes a bit. My steps are like this: original simulated data -> fit flexsurvreg -> resimulate data based on the previous model -> fit flexsurvreg based on resimulated data Should both models have similar estimates since the second data is based on the first fitted model?
Why pars.fmsm
with covariates=0 will have different estimated shapes than print()
and $coefficients
? Shape parameters shouldn't be affected by choice of covariates anyway.
Thanks
The extent of similarity depends on the size of the dataset that you simulate. As the dataset gets larger, the estimates from different simulations will become more consistent with each other and with the true parameter values.
pars.fmsm
with covariates=0 should give the same estimates as the print() output, as long as you are using it on the same model. As I said in my previous comment, I suspected you were running it on a different model.
I checked the models were the same. But, after I simulated more samples and loosened the stopping time of trajectories, the estimates of two models became similar, not the estimated shape from print()
and pars.fmsm
though.
If you suspect functions in package are doing something different from what they are documented to do, you should prepare a minimal example that other people can reproduce (e.g. using the datasets in the package). Otherwise we can only guess what the problem might be.
Hi Chris, I updated the codes in the first post. Since I used simulated data, I attached the codes as well. Sorry if it's too long. Basically, I created a function to generate transition rates using the hazard of log-logistic; a function to generate sojourn time using inverse CDF method, and then the main function to simulate a semi markov, given transition shapes, covariates effects, censoring time etc. I converted the markov chain to survival data using msm2Surv
.
I know in the first model, only the covariates effects covered the true values, which I still haven't figured out. But that should not be the reason that the estimated shapes from print
and parsf.msm
to be so different. Scale with covariates =0 is correct though.
I then resimulated another data (nsample=1000) using sim.fmsm
and refitted another flexsurvreg
. Not just two flexsurvreg
models were different, estimated shapes from parsf.msm
and print
in model 2 were again different. The censoring time for all data was 15. I extended the t
in sim.fmsm
to 20 but censored all observations occurred after t=15.
I'm not sure if this is related. With markov chains, it shouldn't matter if the chain of observations comes from one subject or different subjects, but in survival analysis, people might consider frailty for subject effect. Can flexsurvreg
handle markov chains without including frailty?
I appreciate if you can give me a hint on the discrepancy. Thanks.
That is too long I'm afraid, and will be a lot of work for someone else to debug. I'd advise to focus on one problem at a time, and strip the code down to the absolute minimum necessary needed to diagnose that problem. Hopefully the process of doing that will help you work out where the problems are.
Last comment then, I used the cav data from msm
package and applied the same codes, the estimated shapes (trans 2 or beyond) from pars.fmsm
look like the ones from print
shape est * exp(shape(trans 2 or beyond)). This doesnt hold for the models in my first post though.
Q <- rbind(c(NA,1,2),c(3,NA,4), c(5,6,NA))
cav2 <- msm::cav
cav2 <- cav2[cav2$state!=4,] # remove death status
cav2$age_scale <- as.numeric(scale(cav2$age))
ms <- msm2Surv(cav2, subject='PTNUM', time='years', state="state", Q=Q, cov=c('age_scale','sex'))
ms$trans <- as.factor(ms$trans)
> print(fitlog3)
Call:
flexsurvreg(formula = Surv(time, status) ~ age_scale + sex + shape(trans),
data = ms, dist = "llogis")
Estimates:
data mean est L95% U95% se exp(est) L95% U95%
shape NA 2.5063 2.2884 2.7451 0.1164 NA NA NA
scale NA 4.2797 3.9927 4.5874 0.1516 NA NA NA
age_scale -0.1056 -0.0172 -0.0612 0.0267 0.0224 0.9829 0.9407 1.0271
sex 0.1151 0.1972 0.0358 0.3587 0.0824 1.2180 1.0364 1.4314
shape(trans2) 0.4093 0.4995 0.3850 0.6139 0.0584 1.6478 1.4696 1.8476
shape(trans3) 0.0593 -0.4003 -0.5712 -0.2293 0.0872 0.6701 0.5648 0.7951
shape(trans4) 0.0593 -0.4838 -0.6504 -0.3173 0.0850 0.6164 0.5218 0.7281
shape(trans5) 0.0314 0.1295 -0.1953 0.4544 0.1657 1.1383 0.8226 1.5752
shape(trans6) 0.0314 -0.2930 -0.5386 -0.0474 0.1253 0.7460 0.5835 0.9537
> pars.fmsm(fitlog3, trans=Q, newdata=data.frame(age_scale=0, sex=0, trans=1:6))
$`1`
shape scale
[1,] 2.506347 4.279736
$`2`
shape scale
[1,] 4.13004 4.279736
$`3`
shape scale
[1,] 1.679623 4.279736
$`4`
shape scale
[1,] 1.54493 4.279736
$`5`
shape scale
[1,] 2.852995 4.279736
$`6`
shape scale
[1,] 1.869745 4.279736
Hi, I have a few questions regarding estimating the shape parameter with flexsurvreg and log-logistic distribution. I have a 3 state markov chain, 6 transitions and 2 covariates. Effects of covariates are the same across all transitions. Each transition however has its own shape.
I converted the markov chain to survival data using
msm2Surv
and fitted with flexsurvreg:Except shape (for trans 1), z1, z2 coefficients, all other shape parameters are suppressed to close to 0.
I attempted this command which calculated the shape and scale given covariates. I set covariates=0. These shape parameters are different again, but closer to what I expect. Shape parameters are not affected by covariates, but this command doesn't provide SE nor CI.
I then simulated another data based on the above fitted model using command
sim.fmsm()
. Shape for trans 1, z1, z2 coefficients are almost the same as the above model, but the remaining shape parameters are different from the first model too. How's that possible?Best Regards