editorialbot commented 10 months ago

Submitting author: !--author-handle-->@hkabbech@Kevin-Mattheus-Moerman<!--end-editor-- Reviewers: @imagejan, @ajasja Archive: 10.5281/zenodo.7767749

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/e15328db72dcaf7ca7068f171005ad49"><img src="https://joss.theoj.org/papers/e15328db72dcaf7ca7068f171005ad49/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/e15328db72dcaf7ca7068f171005ad49/status.svg)](https://joss.theoj.org/papers/e15328db72dcaf7ca7068f171005ad49)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@imagejan & @ajasja, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @Kevin-Mattheus-Moerman know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @imagejan

📝 Checklist for @ajasja

editorialbot commented 10 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 10 months ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.03 s (531.6 files/s, 58705.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                           7            109            170            872
Markdown                         3             83              0            152
TeX                              1             15              0             99
YAML                             2              1              3             29
Dockerfile                       1              4              3              6
-------------------------------------------------------------------------------
SUM:                            14            212            176           1158
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 10 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/s41598-019-53663-8 is OK
- 10.1038/s41467-021-26320-w is OK
- 10.1039/C4CP03465A is OK
- 10.1109/TMI.1986.4307764 is OK
- 10.7554/eLife.33125 is OK
- 10.1371/journal.pone.0170165 is OK
- 10.1073/pnas.2104624118 is OK
- 10.1109/ISBI52829.2022.9761672 is OK

MISSING DOIs

- 10.1163/1574-9347_dnp_e612900 may be a valid DOI for title: Keras

INVALID DOIs

- None

editorialbot commented 10 months ago

Wordcount for paper.md is 1068

editorialbot commented 10 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Kevin-Mattheus-Moerman commented 10 months ago

@imagejan, @ajasja this is where the review takes place! This submission has had some delays. I am aware of the holiday period coming up for many. However I would be very grateful if this review could be conducted as streamlined as possible. So please do keep an eye on github notifications from this thread over the coming weeks.

To get started, please call @editorialbot generate my checklist.

Thanks again for your help!

imagejan commented 10 months ago

Review checklist for @imagejan

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/hkabbech/TrackSegNet?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@hkabbech) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

ajasja commented 10 months ago

Review checklist for @ajasja

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/hkabbech/TrackSegNet?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@hkabbech) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Kevin-Mattheus-Moerman commented 9 months ago

@imagejan, @ajasja I hope you are getting on well with the review. Let me know if there are any updates on review progress. Thanks!

Kevin-Mattheus-Moerman commented 9 months ago

@imagejan, @ajasja, 👋 how are you getting on with the review?

Kevin-Mattheus-Moerman commented 9 months ago

@imagejan, @ajasja, 👋 could you please provide an update? Thanks again for helping with this review.

imagejan commented 9 months ago

@Kevin-Mattheus-Moerman sorry, I was delayed by sickness the past two weeks. Will try to finish reviewing this in the next few days.

Kevin-Mattheus-Moerman commented 8 months ago

@imagejan, @ajasja, 👋 thanks again for the help with this review. Are you able to resume at this point?

imagejan commented 8 months ago

I've opened these issues on the TrackSegNet repository:

My main concerns are:

Installation instructions are incomplete for Windows (no GPU used with current installation procedure)
There's no online documentation currently. For example, it would be helpful if the parameter description that is in the paper would also be accessible from the GitHub code repository. Also, the "statement of need" is only present in the paper.
No (rendered) API documentation.
No automated tests.
No documented contribution guidelines.
In the paper, the authors mention related software in the field, but do not describe in detail how their software compares to these other commonly-used packages.

hkabbech commented 8 months ago

Thank you for your inputs @imagejan, I will try to address these issues!

hkabbech commented 7 months ago

Dear @Kevin-Mattheus-Moerman,

I finished implementing the suggestions from @imagejan, the modifications include:

An API documentation (regenerated at every pull request) containing the docstrings general information (introduction, instructions, parameter description as in the paper) and the contribution guidelines: https://tracksegnet.readthedocs.io/en/latest/. (Issue 11)
Updated Windows instructions in the README. (Issue 7)
Creation and publication of a python package in pypi: https://pypi.org/project/tracksegnet/. (Issue 9)
Test of the code quality by running pylint at every commit and test of the code executability on the toy example at every pull request. (Issue 10)

Regarding the related methods mentioned in the field, there are only a few at the moment and the methodologies are quite different from each other. The related software are not always user-friendly, making a fair comparison difficult to achieve, which was also not the initial purpose of this submission.

Kevin-Mattheus-Moerman commented 7 months ago

@imagejan, @ajasja please can you resume (and perhaps finalise if possible) your review at your earliest convenience? Thanks again for your help here!!

Kevin-Mattheus-Moerman commented 7 months ago

@imagejan, @ajasja please can you resume (and perhaps finalise if possible) your review at your earliest convenience? Thanks again for your help here!!

Kevin-Mattheus-Moerman commented 7 months ago

@imagejan, @ajasja :wave:

imagejan commented 6 months ago

All my concerns have been addressed, and I can endorse the paper. Please excuse the delayed response.

@Kevin-Mattheus-Moerman Is there anything else I need to do to finalize the review process?

Kevin-Mattheus-Moerman commented 6 months ago

@imagejan thanks for your help In that case your role here is compete. Thanks again!

Kevin-Mattheus-Moerman commented 6 months ago

@ajasja please can you resume and complete this review at your earliest convenience? Thanks

ajasja commented 6 months ago

@Kevin-Mattheus-Moerman @hkabbech First, I would like to apologize for my slow response.

I have opened the following two minor issues dealing with installation problems: https://github.com/hkabbech/TrackSegNet/issues/24 https://github.com/hkabbech/TrackSegNet/issues/22

My main concerns are:

A short description and links to other tools would be very welcome (no detailed benchmarks, just a paragraph about what the other tools do and how TrackSegNet differs).
In particular, the paper on line 29 the paper mentions major improvements to https://github.com/ismal/DL-MSS, but does not say what these improvements are. In the companion paper (https://academic.oup.com/nar/article/51/20/10992/7288829) the authors go into more detail, which should also be mentioned in this paper:

which major changes include the computation of angles as an additional feature for better distinction of the trajectory confinement, the choice of using the MSD analysis instead of the moment scaling spectrum (MSS) analysis, and the development of a user-friendly python software for replicability on other datasets.

Minor concerns:

It would be great if the raw tiff files were available in the toy examples (and not just the projection). I tried to open the tracks in MTrackJ, but was unable to do so as only one frame was available.
A brief description of the expected output (although it is pretty intuitive).
A mention of supported Python versions (for example, Python 3.11 does not currently work).

This may be outside the scope of the review, but I'm curious -- it seems that the parameters of the states need to be set in advance. Are these just initial values that are adjusted during the training? Probably not, as they seem to be used to create the synthetic trajectories. Is there a way to obtain these parameters from the experimental data? How would one obtain the parameters of the states for an unknown system?

ajasja commented 6 months ago

Also, I forgot one more thing -- since the package is on pip now (which is great), it would be nice is this was mentioned in the installation instructions in the Readme.

Kevin-Mattheus-Moerman commented 5 months ago

@hkabbech can you address this reviewer's comments/feedback? Thanks!

hkabbech commented 5 months ago

Dear all,

Apologies for this late reply, I had other priorities and couldn't find the time to finish this revision.

I have made some changes based on @ajsja comments.

Fixed the open Issues regarding the installation
Precised the supported python version in the README
Added the installation instruction with pip

In paper.md:

Precised the improvements regarding the initial first version of the method
Improved the paragraph part on other tools/method from the literature
Added a brief description of the expected output
Fixed some grammatical mistakes

It would be great if the raw tiff files were available in the toy examples (and not just the projection). I tried to open the tracks in MTrackJ, but was unable to do so as only one frame was available.

I am only in possession of the max. projection of the raw stack images which can be found in the data folder. The raw images would have been too large to upload in GitHub. To preview the trajectories in MTrackJ, you can create a hyperstack in imageJ (image>Hyperstacks>New Hyperstack...) with width=300px, height=50px, channels=1, slices=1 and frames=10,000. This could do the trick to visualize the tracks in time.

This may be outside the scope of the review, but I'm curious -- it seems that the parameters of the states need to be set in advance. Are these just initial values that are adjusted during the training? Probably not, as they seem to be used to create the synthetic trajectories. Is there a way to obtain these parameters from the experimental data? How would one obtain the parameters of the states for an unknown system?

This is somewhat disadvantage of a supervised learning method, the experimental data must be generated based on the experimental data, however not much is known about their dynamics. During my PhD, I developed another method which is unsupervised (no need for synthetic data) based on the noise2noise denoising technique. The description of this other method can be found here, however this method is not yet available as a software tool since it requires more developments and tests from our side.

Thank you for all your inputs!

If there is no more changes to be made, I will update the documentation and the pypi link with a new version.

ajasja commented 5 months ago

@hkabbech Thank you for the changes! Looks good to me @Kevin-Mattheus-Moerman

ajasja commented 5 months ago

@Kevin-Mattheus-Moerman Just checking -- is there anything else that I have to do in the review process?

Kevin-Mattheus-Moerman commented 5 months ago

@ajasja no thanks, all looks good now. Thanks for your help!

Kevin-Mattheus-Moerman commented 5 months ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Double check authors and affiliations (including ORCIDs)
Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
Archive the release on Zenodo/figshare/etc and post the DOI here.
Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.
Make sure that the license listed for the archive is the same as the software license.

Editor Tasks Prior to Acceptance

[x] Read the text of the paper and offer comments/corrections (as either a list or a pull request)
[x] Check that the archive title, author list, version tag, and the license are correct
[x] Set archive DOI with @editorialbot set <DOI here> as archive
[x] Set version with @editorialbot set <version here> as version
[x] Double check rendering of paper with @editorialbot generate pdf
[x] Specifically check the references with @editorialbot check references and ask author(s) to update as needed
[x] Recommend acceptance with @editorialbot recommend-accept

Kevin-Mattheus-Moerman commented 5 months ago

@hkabbech Apologies for the delay over the last two weeks as I was traveling. Given the positive reviews I am now happy to proceed to process this work for acceptance in JOSS. Please can you look at the above post-review checklist and process your steps? Please let me know if you have any questions. Thanks.

Kevin-Mattheus-Moerman commented 5 months ago

@editorialbot check references

editorialbot commented 5 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/s41598-019-53663-8 is OK
- 10.1038/s41467-021-26320-w is OK
- 10.1039/C4CP03465A is OK
- 10.1109/TMI.1986.4307764 is OK
- 10.7554/eLife.33125 is OK
- 10.1371/journal.pone.0170165 is OK
- 10.1073/pnas.2104624118 is OK
- 10.1109/ISBI52829.2022.9761672 is OK

MISSING DOIs

- 10.1163/2214-8647_dnp_e612900 may be a valid DOI for title: Keras

INVALID DOIs

- None

Kevin-Mattheus-Moerman commented 4 months ago

@hkabbech can you work on your points above, and also check that potential DOI issue :point_up:

hkabbech commented 4 months ago

[x] Double check authors and affiliations (including ORCIDs)
[x] Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper. → v1.3.0
[x] Archive the release on Zenodo/figshare/etc and post the DOI here. → 10.5281/zenodo.11448567
[x] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.
[x] Make sure that the license listed for the archive is the same as the software license.

hkabbech commented 4 months ago

@editorialbot check references

editorialbot commented 4 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/s41598-019-53663-8 is OK
- 10.1038/s41467-021-26320-w is OK
- 10.1039/C4CP03465A is OK
- 10.1109/TMI.1986.4307764 is OK
- 10.7554/eLife.33125 is OK
- 10.1371/journal.pone.0170165 is OK
- 10.1073/pnas.2104624118 is OK
- 10.1109/ISBI52829.2022.9761672 is OK

MISSING DOIs

- None

INVALID DOIs

- None

hkabbech commented 4 months ago

Done! The latest version is 1.3.0, and the DOI is 10.5281/zenodo.7767749.

Kevin-Mattheus-Moerman commented 4 months ago

@editorialbot set 10.5281/zenodo.7767749 as archive

editorialbot commented 4 months ago

Done! archive is now 10.5281/zenodo.7767749

Kevin-Mattheus-Moerman commented 4 months ago

@editorialbot set v1.3.0 as version

editorialbot commented 4 months ago

Done! version is now v1.3.0

Kevin-Mattheus-Moerman commented 4 months ago

@editorialbot generate pdf

editorialbot commented 4 months ago

:warning: An error happened when generating the pdf. Author (Hélène Kabbech) is missing affiliation.

Kevin-Mattheus-Moerman commented 4 months ago

@hkabbech looks like this PR might fix the paper compilation error: https://github.com/hkabbech/TrackSegNet/pull/27

hkabbech commented 4 months ago

Sorry about that! Thanks!

hkabbech commented 4 months ago

@editorialbot generate pdf

editorialbot commented 4 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Kevin-Mattheus-Moerman commented 4 months ago

@editorialbot recommend-accept

editorialbot commented 4 months ago

Attempting dry run of processing paper acceptance...

openjournals / joss-reviews

[REVIEW]: TrackSegNet: a tool for trajectory segmentation into diffusive states using supervised deep learning #6157

Status

Reviewer instructions & questions

Checklists

Review checklist for @imagejan

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @ajasja

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance