CamDavidsonPilon / lifelines

Survival analysis in Python
lifelines.readthedocs.org
MIT License
2.34k stars 555 forks source link

Poor results for CoxPHFitter using cubic splines. #1339

Open droully opened 2 years ago

droully commented 2 years ago

I'm using the Cox model with a cubic spline as a semiparametric model and I keep getting poor results, attached is the cumulative hazard which for some reason is decreasing, I've tested using different parameters for the model and even removing all features, but still no good results. Is this a problem with using a spline fitter for the baseline? because I have another model with piecewise baseline which works alright.


naf = NelsonAalenFitter()
naf.fit(train["subscription_age"], event_observed=train["status"])

model = CoxPHFitter("spline",n_baseline_knots=4, penalizer=0.1, l1_ratio=0.2)
model.fit(train, duration_col='subscription_age', event_col='status')
model.print_summary()

ax=naf.plot_cumulative_hazard()
model.baseline_cumulative_hazard_.plot(ax=ax)

image

CamDavidsonPilon commented 2 years ago

Hi @droully, hm you're correct that this doesn't look right. Apriori, it's hard for me to diagnose without having your dataset available. Are you okay sharing (over email?)?

For now, can you post an image of the model.baseline_hazard_?

droully commented 2 years ago

Thank you for the answer, here are both the model.baseline_hazard_ and model.baseline_survival_, they look really bad. I have no problem in sending you the data.

image image

CamDavidsonPilon commented 2 years ago

Definetly shouldn't be negative! Great, send it to me at cam.davidson.pilon@gmail.com