openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
720 stars 38 forks source link

[REVIEW]: TorchSurv: A Lightweight Package for Deep Survival Analysis #7341

Open editorialbot opened 1 week ago

editorialbot commented 1 week ago

Submitting author: !--author-handle-->@melodiemonod<!--end-author-handle-- (Mélodie Monod) Repository: https://github.com/Novartis/torchsurv Branch with paper.md (empty if default branch): 45-joss-submission Version: v0.1.2 Editor: !--editor-->@kanishkan91<!--end-editor-- Reviewers: @WeakCha, @LingfengLuo0510 Archive: Pending

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/02d7496da2b9cc34f9a6e04cabf2298d"><img src="https://joss.theoj.org/papers/02d7496da2b9cc34f9a6e04cabf2298d/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/02d7496da2b9cc34f9a6e04cabf2298d/status.svg)](https://joss.theoj.org/papers/02d7496da2b9cc34f9a6e04cabf2298d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@WeakCha & @LingfengLuo0510, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kanishkan91 know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Checklists

📝 Checklist for @WeakCha

editorialbot commented 1 week ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 week ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.18 s (378.3 files/s, 67362.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          26           1224           2070           4135
Markdown                         7            266              0            681
TeX                              2             47              0            540
R                                7            221            158            520
Jupyter Notebook                 2              0           1551            238
YAML                             6             34              6            201
TOML                             1              8              0             49
Bourne Shell                     4             20              3             40
reStructuredText                 5             16             37             13
make                             1              5              8             10
JSON                             7              0              0              7
-------------------------------------------------------------------------------
SUM:                            68           1841           3833           6434
-------------------------------------------------------------------------------

Commit count by author:

    29  Peter Krusche
    18  Thibaud Coroller
    11  corolth1
     9  melodiemonod
     7  Mélodie Monod
     2  Peter Krusche (Novartis)
     1  Ikko Eltociear Ashimine
editorialbot commented 1 week ago
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.21105/joss.01317 is OK
- 10.32614/CRAN.package.survival is OK
- 10.48550/arXiv.1912.01703 is OK
- 10.5281/zenodo.3352342 is OK
- 10.32614/CRAN.package.survAUC is OK
- 10.1109/cvpr42600.2020.00975 is OK
- 10.1186/s12874-018-0482-1 is OK
- 10.48550/arXiv.2204.07276 is OK
- 10.1007/978-1-4612-4380-9_37 is OK
- 10.1016/s0197-2456(03)00072-2 is OK
- 10.32614/CRAN.package.survival is OK
- 10.32614/CRAN.package.survival is OK
- 10.32614/CRAN.package.survAUC is OK
- 10.32614/CRAN.package.timeROC is OK
- 10.32614/CRAN.package.risksetROC is OK
- 10.32614/CRAN.package.survivalROC is OK
- 10.1093/bioinformatics/btr511 is OK
- 10.32614/CRAN.package.riskRegression is OK
- 10.32614/CRAN.package.SurvMetrics is OK
- 10.32614/CRAN.package.pec is OK
- 10.1111/j.0006-341x.2000.00337.x is OK
- 10.1111/j.0006-341x.2005.030814.x is OK
- 10.1002/bimj.201200045 is OK
- 10.1198/016214507000000149 is OK
- 10.1093/biostatistics/kxy006 is OK
- 10.1002/sim.4154 is OK
- 10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4 is OK
- 10.1002/(sici)1097-0258(19990915/30)18:17/18<2529::aid-sim274>3.0.co;2-5 is OK
- 10.1080/01621459.1977.10480613 is OK
- 10.2307/1402659 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Time-to-Event Prediction with Neural Networks and ...
- No DOI given, and none found for title: The Weibull Distribution

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None
editorialbot commented 1 week ago

Paper file info:

📄 Wordcount for paper.md is 1374

✅ The paper includes a Statement of need section

editorialbot commented 1 week ago

License info:

✅ License found: MIT License (Valid open source OSI approved license)

editorialbot commented 1 week ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

kanishkan91 commented 1 week ago

@melodiemonod, @WeakCha, @LingfengLuo0510, This is the review thread for the paper. All of our communications will happen here from now on.

Please read the "Reviewer instructions & questions" in the first comment above.

For @WeakCha, @LingfengLuo0510 - Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.

As you are probably already aware, The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention https://github.com/openjournals/joss-reviews/issues/7341 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule.

Thanks in advance and let me know if you have any questions!!

WeakCha commented 6 days ago

Review checklist for @WeakCha

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

editorialbot commented 6 days ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

WeakCha commented 3 days ago

@melodiemonod @kanishkan91

Thank you so much for this package, this work looks super interesting! Here are my initial comments:

For the paper:

  1. Adding an example after you introduce the functionality would be great. This example does not need to be comprehensive, but needs to be executable without any error. Your tutorial has done a great job on this, but people may refer to your paper first and want to seek for something that is easier to use.
  2. There are some typos in your paper. For example, scitkit-survival should be scikit-survival in Figure 1, MyPyTorcWeibullhModel should be MyPyTorchWeibullModel (between line 46 and 47), and MyPyTorchXCoxModel should be MyPyTorchCoxModel (between line 51 and 52).

For your tutorial (https://opensource.nibr.com/torchsurv/notebooks/introduction.html), could you please show what helpers_introduction is? I know there should be some plugins but I cannot find them, and I will give further comments after I successfully reproduce your tutorial.

Thank you so much for your work! I am looking forward to reviewing your updated paper and package!

tcoroller commented 10 hours ago

Dear @WeakCha,

Thank you for your early comments. Regarding your last one (helpers_*), would it be ok with you if we add this line below in the notebooks with the link to the helper file itself?

For the first notebook (introduction), the helpers_introduction line will be the following:

# PyTorch boilerplate - see https://github.com/Novartis/torchsurv/blob/main/docs/notebooks/helpers_introduction.py
from helpers_introduction import Custom_dataset, plot_losses

I would discourage adding the code directly in the notebook because it would overcrowd the notebook:

  1. I would need to be at the beginning of the notebook to be properly loaded and run, which is annoying because one helper (helpers_momentum) is ~200lines of code
  2. They are simply boilerplate PyTorch code, with no relationship to TorchSurv and well established function (model and datasets, plot losses).

If our suggestion isn't satisfying, I can look into other option (one could be to have the cell hidden, but requires another package thus making it rely on more dependancies)