search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[REVIEW]: TelescopeML -- I. An End-to-End Python Package for Interpreting Telescope Datasets through Training Machine Learning Models, Generating Statistical Reports, and Visualizing Results #6346

Closed editorialbot closed 3 months ago

editorialbot commented 9 months ago

Submitting author: !--author-handle-->@EhsanGharibNezhad<!--end-author-handle-- (Ehsan Gharib-Nezhad) Repository: https://github.com/EhsanGharibNezhad/TelescopeML Branch with paper.md (empty if default branch): joss Version: 0.0.5 Editor: !--editor-->@plaplant<!--end-editor-- Reviewers: @oparisot, @mwalmsley Archive: 10.5281/zenodo.11553655

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/341c09d394beb3918d85a6b176eb9b83"><img src="https://joss.theoj.org/papers/341c09d394beb3918d85a6b176eb9b83/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/341c09d394beb3918d85a6b176eb9b83/status.svg)](https://joss.theoj.org/papers/341c09d394beb3918d85a6b176eb9b83)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@oparisot, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @plaplant know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @oparisot

📝 Checklist for @mwalmsley

editorialbot commented 9 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 9 months ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.13 s (258.7 files/s, 137169.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          11            952           1486           2122
Jupyter Notebook                 4              0          10305            755
YAML                             6             10              9            753
TeX                              1             35              0            384
Markdown                         2             60              0            228
reStructuredText                 7            135             79            127
DOS Batch                        1              8              1             26
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                            33           1204          11887           4404
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 9 months ago

Wordcount for paper.md is 1153

editorialbot commented 9 months ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.3847/1538-3881/aae77c is OK
- 10.3847/2041-8213/acb04a is OK
- 10.1038/s41586-022-05591-3 is OK
- 10.1088/1538-3873/acd1b5 is OK
- 10.48550/arXiv.1705.05165 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.21105/joss.03021 is OK
- 10.1109/MCSE.2007.55 is OK
- 10.1038/s41586-020-2649-2 is OK
- 10.5281/zenodo.3509134 is OK
- 10.3847/1538-4357/ac7c74 is OK
- 10.1146/annurev-astro-082214-122522 is OK
- 10.3847/1538-4357/acabc2 is OK
- 10.21105/joss.04873 is OK
- 10.3847/1538-4357/ace530 is OK

MISSING DOIs

- 10.1088/0004-637x/783/2/70 may be a valid DOI for title: A systematic retrieval analysis of secondary eclipse spectra. II. A uniform analysis of nine planets and their C to O ratios
- 10.3847/1538-4357/ab1b51 may be a valid DOI for title: Exoplanet reflected-light spectroscopy with PICASO
- 10.3847/1538-4357/ac4399 may be a valid DOI for title: ExoMiner: A Highly Accurate and Explainable Deep Learning Classifier that Validates 301 New Exoplanets

INVALID DOIs

- https://doi.org/10.1016/j.ymssp.2020.107398 is INVALID because of 'https://doi.org/' prefix
editorialbot commented 9 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

plaplant commented 9 months ago

@oparisot thanks for agreeing to review! To generate your reviewer checklist, in this thread please comment:

@editorialbot generate my checklist

Then when you get a chance, please begin making your way through the checklist. Let me know if you have any questions!

oparisot commented 9 months ago

Review checklist for @oparisot

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

EhsanGharibNezhad commented 9 months ago

@plaplant, thanks again for assigning a reviewer, and @oparisot, thank you for dedicating your time to review this package!

@oparisot I'll make sure to check and address all your comments, but please don't hesitate to reach out if you have any questions or share any instructions, as this is my first JOSS submission and experience with this process.

oparisot commented 9 months ago

Hello @EhsanGharibNezhad,

I will go deeply into the review, but before, you should add a short summary in the paper, as required in JOSS guidelines: "A summary describing the high-level functionality and purpose of the software for a diverse, non-specialist audience."

Moreover, you should clarifiy the 'Functionality and Key Features'. The current paper proposes a high-level list of features, but it is too vague. You should describe the inputs and outputs (files formats? time series?), you should present the goal of AI models (what do you mean by 'predicting atmospheric parameters'?),you should describe the type of CNN, etc.

Finally, you should position and compare your software to existing tools (ex https://github.com/MartianColonist/POSEIDON, if it not appropriate then please list tools that are 'comparable' to your software.

I will test the software in few days.

Thank you,

Olivier

EhsanGharibNezhad commented 8 months ago

Hi @oparisot, Thanks for the feedback. I'll make sure to address your comments within the next few days and find the softwares that I could compare my package with.

plaplant commented 8 months ago

@editorialbot add @mwalmsley as reviewer

editorialbot commented 8 months ago

@mwalmsley added to the reviewers list!

EhsanGharibNezhad commented 8 months ago

Hi Oliver @oparisot:

To address your comments, I added the following sections:

I also modified the Functionality and Key Features section to clearly list the tasks. Besides, I updated the DOIs suggested earlier. Please let me know if it still looks vague. Thank you!

EhsanGharibNezhad commented 7 months ago

Hi Paul @plaplant, Just making sure I'm not missing anything, all the comments will be listed here in the GitHub thread, right?

plaplant commented 7 months ago

@EhsanGharibNezhad for comments on the submission, we leave that up to the reviewers' preference. Some will make comments in this review thread, and others will make issues on the software repository and reference them as part of this thread, especially if the comments are more involved or there are several issues that need to be addressed. Either way, at least a reference should appear in this thread.

@oparisot thanks very much for your initial review! When you get a chance, please provide any additional feedback you have, either as a comment on this thread or by making an issue on the software repository. Thanks!

@mwalmsley when you get a chance, please make the following comment in this thread:

@editorialbot generate my checklist

Then please start going through the review elements contained in the checklist. Let me know if you have any questions!

oparisot commented 7 months ago

Hi @EhsanGharibNezhad,

Thank you for the update of the paper. Now, I'm testing the software -- I will give you my feedback.

Ideas to improve the repository: can you add a short 'contribution guide' (ex: https://contributing.md/example/)?

Olivier

oparisot commented 7 months ago

The software is easy to use by following the instructions, and the use-cases are well explained. I'm in favour of accepting the paper, if guidelines (even slight ones) for contribution / support are added in the git repository.

Olivier

EhsanGharibNezhad commented 7 months ago

Hi @EhsanGharibNezhad,

Thank you for the update of the paper. Now, I'm testing the software -- I will give you my feedback.

Ideas to improve the repository: can you add a short 'contribution guide' (ex: https://contributing.md/example/)?

Olivier

Hi Oliver @oparisot, This is a great comment. Thanks for sharing this link. I added two docs to the main branch to address that:

  1. CODE_OF_CONDUCT.md: https://github.com/EhsanGharibNezhad/TelescopeML/blob/main/CODE_OF_CONDUCT.md
  2. CONTRIBUTING.md: https://github.com/EhsanGharibNezhad/TelescopeML/blob/main/CONTRIBUTING.md
oparisot commented 7 months ago

@EhsanGharibNezhad I've tested the software and it works well - the use cases are well described in the documentation and run efficiently. Great job! @plaplant It's ok for me -- I am in favour of acceptance.

plaplant commented 7 months ago

@oparisot thanks for your review! If you have completed all components of the reviewer checklist, please tick the boxes so we can consider the review "complete".

@mwalmsley when you get a chance, please comment @editorialbot generate my checklist and begin working your way through the elements on the list. Thanks!

EhsanGharibNezhad commented 7 months ago

@EhsanGharibNezhad I've tested the software and it works well - the use cases are well described in the documentation and run efficiently. Great job! @plaplant It's ok for me -- I am in favour of acceptance.

Great! Thank you Oliver for taking the time and testing the package.

mwalmsley commented 6 months ago

Review checklist for @mwalmsley

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

mwalmsley commented 6 months ago

@editorialbot generate pdf

(I don't see the new sections on the link above so I assume a new pdf is needed)

editorialbot commented 6 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mwalmsley commented 6 months ago

Hello. I am sorry it has taken me some time to get to this. From a look at the documentation and the paper, this is clearly a substantial piece of scholarly work and will benefit the field.

It's not easy for me to try the software right now because I'm traveling and won't be back for another two weeks. Ping me if you need a faster review than that for any reason and I'll give it a try earlier.

(my one very unhelpful comment is that TelescopeML is a rather ambiguous name. Many astroML projects also use telescopes ;) )

EhsanGharibNezhad commented 6 months ago

Hi Mike, Thanks for the heads-up. Two weeks should be okay as we are currently pushing a new version of the package and updating the Sphinx tutorials. Regarding the name, I agree that it is pretty general and it can be ambiguous, but this is a long-term program which includes different data analysis problems related to spectral analysis and interpretation of observational data.

mwalmsley commented 6 months ago

Hi @EhsanGharibNezhad . I'm working through the installation and found the following error with conda:

ResolvePackageNotFound: 
  - pyobjc-core=9.2
  - pyobjc-framework-cocoa=9.2
  - libgfortran=5.0.0

From a quick look, I think the conda forge version of pyobjc-core is only available on osx (see the installers tags here. Similarly libgfortran is available on all platforms but 5.0.0 only seems to have osx installers here.

Could you please update the .yml? Then I will reinstall and run the notebooks, and then approve :)

Cheers, Mike

EhsanGharibNezhad commented 5 months ago

Hi Mike @mwalmsley Thanks for the comments. I'll address those within the next few days or a week if you'd not mind and will get back to you later.

plaplant commented 5 months ago

@oparisot thank you very much for submitting your review! I'm glad to hear that you feel this is ready for acceptance.

@EhsanGharibNezhad @mwalmsley thanks for keeping up with the review! It sounds like there is just this one small issue about the package's environment.yml file until we're ready to move forward with acceptance. Please let me know if there is anything I can do to help move this forward.

oparisot commented 5 months ago

You're welcome!

EhsanGharibNezhad commented 5 months ago

Hi All,

Since the last edition, the following key updates have been made to the package and tutorials:

  1. Notebooks now include flowcharts and figures, along with more detailed descriptions to provide a clear roadmap.
  2. Function names in the modules are now more explicit, and some have been edited.
  3. The online Sphinx documentation has been revised.
  4. A new version of TelescopeML (v0.0.5) is now available on PyPI.
  5. The environment file has been updated following @mwalmsley 's suggestions.

Additionally, one of my team members, Mahdi Habibi (https://github.com/mdhabibi), dedicated his time to editing the notebooks and assisting with code testing and compatibility. As a result, his name has been added to the JOSS paper draft as a co-author.

Please let me know if you have any questions. Thank you!

plaplant commented 5 months ago

@EhsanGharibNezhad thank you for the update! I appreciate the information. I think adding @mdhabibi as a co-author is okay if they have contributed in the way you say, so please add them to the co-authors if you haven't yet.

@mwalmsley when you get a chance, please take a quick look at the latest version. If you are happy with the changes, please update your review checklist to indicate any outstanding issues have been addressed. Thanks!

plaplant commented 5 months ago

@editorialbot generate pdf

editorialbot commented 5 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mwalmsley commented 5 months ago

Happy with the changes and happy to recommend acceptance. Thanks for fixing the install!

plaplant commented 5 months ago

@mwalmsley thank you for your review! Glad to hear that you recommend acceptance.

@EhsanGharibNezhad now that both reviewers have recommended acceptance, we can move forward with the final checks before publication. In looking at the manuscript proof from a few days ago, it seems like Mahdi Habibi has not been added as an author. Please update the author list when you get a chance.

In addition to adding the author, please confirm that the latest version of TelescopeML is v0.0.5 (from your above comment). Also, please archive the software on Zenodo, figshare, or some other long-term hosting platform. Once that is done, please post the DOI for the package here. Let me know if you have any questions. Thanks!

plaplant commented 5 months ago

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

Editor Tasks Prior to Acceptance

EhsanGharibNezhad commented 5 months ago

Dear All- Thank you so much for the time you dedicated to improving this paper and the package. I made sure to acknowledge your names in the JOSS paper draft for your insightful comments.

Hi Paul, @plaplant, I addressed all these comments and archived the repo/package in Zenodo:

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

  • Double check authors and affiliations (including ORCIDs) - Done! ✅

  • Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper. - Done! ✅ Version number: 0.0.5

  • Archive the release on Zenodo/figshare/etc and post the DOI here.- Done! ✅ 10.5281/zenodo.11553655 Link: https://zenodo.org/records/11553655

  • Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.- Done! ✅

  • Make sure that the license listed for the archive is the same as the software license.- Done! ✅

EhsanGharibNezhad commented 5 months ago

@plaplant, I have two questions:

  1. We are about to submit a scientific paper to the ApJ journal that discusses the implications of this package and the trained CNN models on brown dwarf spectra. Could you please instruct how to address that in the JOSS paper? Suggested text: "This article and software are linked with the research article DOI LINK OF ApJ PAPER - TBD, published in The Astrophysical Journal." The title of the paper is "TelescopeML – II. Convolutional Neural Networks for Predicting Brown Dwarf Atmospheric Parameters"

  2. The footnote in the rendered PDF does not show my full name correctly. It should be "Ehsan (Sam) Gharib-Nezhad et al. (2024)....". Could you please let me know how to fix this?

plaplant commented 5 months ago

@editorialbot generate pdf

editorialbot commented 5 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

plaplant commented 4 months ago

@editorialbot set 10.5281/zenodo.11553654 as archive

editorialbot commented 4 months ago

Done! archive is now 10.5281/zenodo.11553654

plaplant commented 4 months ago

@editorialbot set 10.5281/zenodo.11553655 as archive

editorialbot commented 4 months ago

Done! archive is now 10.5281/zenodo.11553655

plaplant commented 4 months ago

@editorialbot set 0.0.5 as version

editorialbot commented 4 months ago

Done! version is now 0.0.5

plaplant commented 4 months ago

@EhsanGharibNezhad thanks for your patience and your work on getting this over the finish line! I made a few suggestions for changes to the paper in a PR (https://github.com/EhsanGharibNezhad/TelescopeML/pull/230). To answer your questions above:

  1. We are about to submit a scientific paper to the ApJ journal that discusses the implications of this package and the trained CNN models on brown dwarf spectra. Could you please instruct how to address that in the JOSS paper? Suggested text: "This article and software are linked with the research article DOI LINK OF ApJ PAPER - TBD, published in The Astrophysical Journal." The title of the paper is "TelescopeML – II. Convolutional Neural Networks for Predicting Brown Dwarf Atmospheric Parameters"

JOSS does have a partnership with AAS publishing for companion papers, which is outlined more here. My reading of this partnership is that the joint publication should have been established earlier in the review process, before the review of the JOSS paper began so that the reviewers could be made aware. The metadata tags aas-doi and aas-journal are used for this type of joint publication, so I think they do not apply in this case and should probably be removed from the paper.

Would it be possible to submit your article to a pre-print server (e.g., arXiv) and link the DOI to that? If you would prefer to wait for the DOI of the published ApJ article, we would most likely have to pause the publication of the JOSS paper until the ApJ article has been accepted. Please let me know how you'd like to proceed.

  1. The footnote in the rendered PDF does not show my full name correctly. It should be "Ehsan (Sam) Gharib-Nezhad et al. (2024)....". Could you please let me know how to fix this?

The PR I opened on the paper has a suggestion that should fix this. Thanks for pointing it out!

Let me know if you have any questions!

EhsanGharibNezhad commented 4 months ago

10.5281/zenodo.11553655

@EhsanGharibNezhad thanks for your patience and your work on getting this over the finish line! I made a few suggestions for changes to the paper in a PR (EhsanGharibNezhad/TelescopeML#230). To answer your questions above:

  1. We are about to submit a scientific paper to the ApJ journal that discusses the implications of this package and the trained CNN models on brown dwarf spectra. Could you please instruct how to address that in the JOSS paper? Suggested text: "This article and software are linked with the research article DOI LINK OF ApJ PAPER - TBD, published in The Astrophysical Journal." The title of the paper is "TelescopeML – II. Convolutional Neural Networks for Predicting Brown Dwarf Atmospheric Parameters"

JOSS does have a partnership with AAS publishing for companion papers, which is outlined more here. My reading of this partnership is that the joint publication should have been established earlier in the review process, before the review of the JOSS paper began so that the reviewers could be made aware. The metadata tags aas-doi and aas-journal are used for this type of joint publication, so I think they do not apply in this case and should probably be removed from the paper.

I removed the metadata tags aas-doi and aas-journal.

Would it be possible to submit your article to a pre-print server (e.g., arXiv) and link the DOI to that? If you would prefer to wait for the DOI of the published ApJ article, we would most likely have to pause the publication of the JOSS paper until the ApJ article has been accepted. Please let me know how you'd like to proceed.

No, I cannot post it on arXiv until I receive the reviewers' comments. I have added this following statement in the summary, but please feel free to remove it if you think it is unnecessary. Regardless, please proceed with the publication of the JOSS paper without waiting for The Astrophysical Journal.

The implications and scientific outcomes from the trained CNN models and this package are under revision for The Astrophysical Journal under the title TelescopeML – II: Convolutional Neural Networks for Predicting Brown Dwarf Atmospheric Parameters.

  1. The footnote in the rendered PDF does not show my full name correctly. It should be "Ehsan (Sam) Gharib-Nezhad et al. (2024)....". Could you please let me know how to fix this?

The PR I opened on the paper has a suggestion that should fix this. Thanks for pointing it out!

Let me know if you have any questions!

Fixed! Thanks!

EhsanGharibNezhad commented 4 months ago

@plaplant

Please find the new version of the JOSS draft with the following changes: