[REVIEW]: ProgPy: Python Packages for Prognostics and Health Management of Engineering Systems

[editorialbot]

Submitting author: !--author-handle-->@teubert<!--end-author-handle-- (Chris Teubert) Repository: https://github.com/nasa/prog_models Branch with paper.md (empty if default branch): paper Version: v1.5 Editor: !--editor-->@kellyrowland<!--end-editor-- Reviewers: @tbsexton, @nkrusch Archive: 10.5281/zenodo.8097013

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/86cc587435ae9f766eb67e0b22d49a67"><img src="https://joss.theoj.org/papers/86cc587435ae9f766eb67e0b22d49a67/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/86cc587435ae9f766eb67e0b22d49a67/status.svg)](https://joss.theoj.org/papers/86cc587435ae9f766eb67e0b22d49a67)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@matthewjdaigle & @tbsexton, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kellyrowland know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @tbsexton

📝 Checklist for @nkrusch

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=0.25 s (544.0 files/s, 146713.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                      12           2769           3087          11446
Python                          80           2196           3692           8265
CSS                              5            392             76           1505
HTML                             8            282              0           1349
YAML                             8             55             68            297
Markdown                         8             76              0            293
Jupyter Notebook                 1              0            784            181
TeX                              1              8              0             55
reStructuredText                14             17             17             40
-------------------------------------------------------------------------------
SUM:                           137           5795           7724          23431
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 1142

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.36001/ijphm.2017.v8i2.2618 may be a valid DOI for title: A Generic Software Architecture for Prognostics (GSAP)
- 10.36001/phmconf.2022.v14i1.3238 may be a valid DOI for title: Enabling in-time prognostics with surrogate modeling through physics-enhanced Dynamic Mode Decomposition method

INVALID DOIs

- None

[matthewjdaigle]

Review checklist for @matthewjdaigle

Conflict of interest

• [ ] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [ ] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [ ] Repository: Is the source code for this software available at the https://github.com/nasa/prog_models?
• [ ] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [ ] Contribution and authorship: Has the submitting author (@teubert) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [ ] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• [ ] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• [ ] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• [ ] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• [ ] Installation: Does installation proceed as outlined in the documentation?
• [ ] Functionality: Have the functional claims of the software been confirmed?
• [ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [ ] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [ ] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [ ] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [ ] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [ ] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• [ ] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [ ] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [ ] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[matthewjdaigle]

@kellyrowland Regarding COI policy, I'm a former colleague with the submitting author (5+ years ago), however, within the repository's readme I am credited as an author, since it is based on my previous work (which is also cited in the paper). So there is definitely a perception of a COI here. Let me know how to proceed.

[rtbs-dev]

Review checklist for @tbsexton

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the https://github.com/nasa/prog_models?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@teubert) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• [x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• [x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• [x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[kellyrowland]

@matthewjdaigle thanks for pointing that out - in the interest of impartiality it would probably be best that I remove you from the reviewer list. My apologies for not catching this in the pre-review stage.

Would you be able to recommend someone in the broader field who would be a good fit for reviewing this?

[matthewjdaigle]

@matthewjdaigle thanks for pointing that out - in the interest of impartiality it would probably be best that I remove you from the reviewer list. My apologies for not catching this in the pre-review stage.

Would you be able to recommend someone in the broader field who would be a good fit for reviewing this?

I'll check with my network and let you know.

[matthewjdaigle]

@kellyrowland I can recommend @samim91 as a reviewer.

[kellyrowland]

Thanks very much - @samim91 would you be interested in and available for reviewing this submission to JOSS?

@tbsexton thanks for getting started on your review - feel free to reach out if you have any questions.

[samim91]

Hi Kelly,

Yes, would be happy to. Can you include my stanford address in copy? @.***

Best, Samim

Sent from Outlook for iOShttps://aka.ms/o0ukef

---

From: Kelly L. Rowland @.> Sent: Monday, February 13, 2023 9:49:19 AM To: openjournals/joss-reviews @.> Cc: samim91 @.>; Mention @.> Subject: Re: [openjournals/joss-reviews] [REVIEW]: ProgPy: Python Packages for Prognostics and Health Management of Engineering Systems (Issue #5099)

Thanks very much - @samim91https://github.com/samim91 would you be interested in and available for reviewing this submission to JOSS?

@tbsextonhttps://github.com/tbsexton thanks for getting started on your review - feel free to reach out if you have any questions.

— Reply to this email directly, view it on GitHubhttps://github.com/openjournals/joss-reviews/issues/5099#issuecomment-1428394130, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKSWKEL3LHAANS5LCT6OVOTWXJXZ7ANCNFSM6AAAAAAUFUHRZ4. You are receiving this because you were mentioned.Message ID: @.***>

[kellyrowland]

Hi @samim91 - thanks for the response and interest here. I think you may want to add your Stanford email address onto your Github profile to have it copied on messages; I'm not sure that I can ping it via commenting on the issue from the Github website.

The review process is done here in the Github issue; you'll generate a checklist and then check off those items as you progress through your review. Please let me know if you have any questions or concerns.

[kellyrowland]

@editorialbot add @samim91 as reviewer

[editorialbot]

@samim91 added to the reviewers list!

[teubert]

Hello @samim91, @tbsexton, Thank you both so much for reviewing this paper.

I wanted to check in- do you need anything from me at this point for the review?

[kellyrowland]

@editorialbot remove @matthewjdaigle from reviewers

[editorialbot]

@matthewjdaigle removed from the reviewers list!

[kellyrowland]

@tbsexton @samim91 please let me know whether or not you think you'll be able to make progress on your reviews in the near future so that I can start reaching out to other potential reviewers if needed.

[rtbs-dev]

Definitely, just had some obligations come up recently but I should get through a good amount this week

[samim91]

I will not be able to conduct the review without further changes to my system. It wouldn't be a bad idea to reach out to other potential reviewers.

[rtbs-dev]

Link to issue of notes related to my review.

[rtbs-dev]

@teubert Outstanding checklist items:

• Does the list of citations include reference for the datasets you have bundled with ProgPy? I couldn't find a relevant one.

• I don't see reference to state-of-the-community regarding existing packages, libraries, or comparisons benchmarks. Obviously this is a bit of a difficulty when there's not a huge python PHM community, but there most be something out there, e.g. used in courses? Could also reference:

  • Lifelines which gets used for survival analysis in maintenance contexts
  • pomegranate which has modular systems for stochastic process estimation, markov chains, bayes nets, HMMs, etc, all commonly used in PHM.
  • Sklearn which is the backbone of most academic courses I've seen, e.g. for anomaly detection (despite not at all being meant for PHM specicifally, but nonetheless)
  • E.g. Simantha Isn't specifically a tool for PHM, and it's not nearly as close to "polished & ready" as ProgPy (iirc), but I know it has a plugin system for external degradation estimation when running the manufacturing sim. Maybe it's an opportunity for ProgPy models to be used? There's probably lots of examples like this! Would make the paper easier to relate to.

  Mentioning some things like this and discussing how prog_models fits relative to them would go a ways to say "how are people doing this now, and how can they do it better with ProgPy?". I know you all say "ad hoc" but ad hoc with/using what?

[kellyrowland]

@editorialbot remove @samim91 from reviewers

[editorialbot]

@samim91 removed from the reviewers list!

[kellyrowland]

Hi @nkrusch @arunmano121 👋 would you be interested in and available to review this JOSS submission?

[arunmano121]

My apologies that I will not be able to review due to upcoming travel commitments.

Best regards, Arun Manohar

-- Sent from my phone

On Mar 27, 2023, at 11:02 AM, Kelly L. Rowland @.***> wrote:



Hi @nkruschhttps://github.com/nkrusch @arunmano121https://github.com/arunmano121 👋 would you be interested in and available to review this JOSS submission?

— Reply to this email directly, view it on GitHubhttps://github.com/openjournals/joss-reviews/issues/5099#issuecomment-1485590437, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATAHAZSTRWYAJESDYRLEVFLW6HI2BANCNFSM6AAAAAAUFUHRZ4. You are receiving this because you were mentioned.Message ID: @.***>

[nkrusch]

Hi @nkrusch ... would you be interested in and available to review this JOSS submission?

What is the timeline? I should have sufficient availability next week.

[teubert]

@teubert Outstanding checklist items:

• Does the list of citations include reference for the datasets you have bundled with ProgPy? I couldn't find a relevant one.

• I don't see reference to state-of-the-community regarding existing packages, libraries, or comparisons benchmarks. Obviously this is a bit of a difficulty when there's not a huge python PHM community, but there most be something out there, e.g. used in courses? Could also reference:

  • Lifelines which gets used for survival analysis in maintenance contexts
  • pomegranate which has modular systems for stochastic process estimation, markov chains, bayes nets, HMMs, etc, all commonly used in PHM.
  • Sklearn which is the backbone of most academic courses I've seen, e.g. for anomaly detection (despite not at all being meant for PHM specicifally, but nonetheless)
  • E.g. Simantha Isn't specifically a tool for PHM, and it's not nearly as close to "polished & ready" as ProgPy (iirc), but I know it has a plugin system for external degradation estimation when running the manufacturing sim. Maybe it's an opportunity for ProgPy models to be used? There's probably lots of examples like this! Would make the paper easier to relate to.

  Mentioning some things like this and discussing how prog_models fits relative to them would go a ways to say "how are people doing this now, and how can they do it better with ProgPy?". I know you all say "ad hoc" but ad hoc with/using what?

Hello @tbsexton,

Thanks for your comments here and the recommendations you put in an issue. I added references for the datasets (also put out by my team) which progpy provides access to. I also added a few more sentences on other packages, including the 4 you mentioned.

[kellyrowland]

@nkrusch ideally reviewers complete their reviews within 6 weeks; please let me know if that sounds feasible on your end and I'll add you to the reviewer list if so.

[nkrusch]

@kellyrowland that timeline sounds feasible to me.

[kellyrowland]

Great.

@editorialbot add @nkrusch to reviewers

[kellyrowland]

@editorialbot add @nkrusch to reviewers

[editorialbot]

@nkrusch added to the reviewers list!

[kellyrowland]

Hi @nkrusch 👋 ping on this review issue when you're able to get started with it. If you have any questions or need to set it down, please let me know.

[nkrusch]

Hi @kellyrowland, I plan to start next week (end of week). I will review in detail all instructions at that time. I will let you know if I have questions, thank you.

[nkrusch]

Review checklist for @nkrusch

Conflict of interest

• [x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• [x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• [x] Repository: Is the source code for this software available at the https://github.com/nasa/prog_models?
• [x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• [x] Contribution and authorship: Has the submitting author (@teubert) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• [x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
• [x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
• [x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
• [x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

• [x] Installation: Does installation proceed as outlined in the documentation?
• [x] Functionality: Have the functional claims of the software been confirmed?
• [x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• [x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• [x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• [x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• [x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• [x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• [x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• [x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• [x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
• [x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
• [x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• [x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[nkrusch]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[nkrusch]

@editorialbot check references

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7329096 is OK

MISSING DOIs

- 10.36001/ijphm.2017.v8i2.2618 may be a valid DOI for title: A Generic Software Architecture for Prognostics (GSAP)
- 10.36001/phmconf.2022.v14i1.3238 may be a valid DOI for title: Enabling in-time prognostics with surrogate modeling through physics-enhanced Dynamic Mode Decomposition method

INVALID DOIs

- None

[nkrusch]

Link to my review comments -- suggestions only, the checklist is satisfactorily met.

[nkrusch]

@kellyrowland I have completed my review. Let me know if additional action is required.

[teubert]

Thanks @nkrusch

@kellyrowland I've resolved all of the review comments. What are the next steps?

[kellyrowland]

@teubert great! Apologies for the delay on my end.

At this point I will ask you to issue a new tagged release of the software (if changed since the start of the review), and archive it (on Zenodo, figshare, or elsewhere). Then, please then post the version number and archive DOI here in the review issue, and I'll follow some subsequent wrap-up steps from there.

[teubert]

Thanks @kellyrowland - we're planning a release for week of 6/19. I'll send a message once that's complete and archived.

[kellyrowland]

Excellent, I'll be on the lookout for the note.

[kellyrowland]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left: