search

havakv / pycox

Survival analysis with PyTorch
BSD 2-Clause "Simplified" License
780 stars 180 forks source link

Reproduction of the results in JMLR19 paper #164

Open minxueric opened 1 year ago

minxueric commented 1 year ago

I find the results in the examples notebook are quite different from the results shown in your JMLR2019 paper.

For example, the Cox-time method on the metabric dataset: in paper: Concordance=0.662, Integrated Brier score=0.172(best), Integrated binomial log-likelihood=-0.511 in notebook: Concordance=0.675, Integrated Brier score=0.159, Integrated binomial log-likelihood=-0.470

Could you please explain the difference and give the experiment code for reproduction of the results in the paper?

havakv commented 1 year ago

The examples here are just examples with rather arbitrary hyperparameters. In the paper we have a common hyperparameter tuning strategy all methods we compare, to make the comparison fair (it's very easy to overtfit to such small datasets by just trying some configurations by hand). This is listed in section 6.1.1 of the paper. If you want to reproduce the results, you would need to follow the experiments as explained there.