search

pyOpenSci / software-submission

Submit your package for review by pyOpenSci here! If you have questions please post them here: https://pyopensci.discourse.group/
94 stars 36 forks source link

THzTools submission #209

Open jsdodge opened 3 months ago

jsdodge commented 3 months ago

Submitting Author: Name (@jsdodge) All current maintainers: (@jsdodge) Package Name: THzTools Data analysis software tools for terahertz time-domain spectroscopy (THz-TDS) Repository Link: https://github.com/dodge-research-group/thztools Version submitted: 0.5.4 EiC: @cmarmo Editor:@banesullivan
Reviewer 1: @frank1010111
Reviewer 2: @Romain-Peretti
Archive: TBD JOSS DOI: TBD Version accepted: TBD Date accepted (month/day/year): TBD

Code of Conduct & Commitment to Maintain Package

Description

THzTools provides tools to simplify and improve procedures for data analysis in terahertz time-domain spectroscopy (THz-TDS). Some of the methods included in the package were described previously in the paper at this link. As the name suggests, terahertz time-domain spectroscopy involves measurements of terahertz-frequency electromagnetic waveforms that are are acquired as a function of time. A variety of methods exist to transform these measurements into functions of frequency, but the standard procedures have several pitfalls. THzTools provides software tools that make it easier for researchers to use the best available methods for analyzing their data.

Scope

Domain Specific

Community Partnerships

If your package is associated with an existing community please check below:

[^1]: Please fill out a pre-submission inquiry before submitting a data visualization package.

Technical checks

For details about the pyOpenSci packaging requirements, see our packaging guide. Confirm each of the following by checking the box. This package:

Publication Options

JOSS Checks - [x] The package has an **obvious research application** according to JOSS's definition in their [submission requirements][JossSubmissionRequirements]. Be aware that completing the pyOpenSci review process **does not** guarantee acceptance to JOSS. Be sure to read their submission requirements (linked above) if you are interested in submitting to JOSS. - [x] The package is not a "minor utility" as defined by JOSS's [submission requirements][JossSubmissionRequirements]: "Minor ‘utility’ packages, including ‘thin’ API clients, are not acceptable." pyOpenSci welcomes these packages under "Data Retrieval", but JOSS has slightly different criteria. - [x] The package contains a `paper.md` matching [JOSS's requirements][JossPaperRequirements] with a high-level description in the package root or in `inst/`. - [x] The package is deposited in a long-term repository with the DOI: https://zenodo.org/doi/10.5281/zenodo.10100093 *Note: JOSS accepts our review as theirs. You will NOT need to go through another full review. JOSS will only review your paper.md file. Be sure to link to this pyOpenSci issue when a JOSS issue is opened for your package. Also be sure to tell the JOSS editor that this is a pyOpenSci reviewed package once you reach this step.*

Are you OK with Reviewers Submitting Issues and/or pull requests to your Repo Directly?

This option will allow reviewers to open smaller issues that can then be linked to PR's rather than submitting a more dense text based review. It will also allow you to demonstrate addressing the issue via PR links.

Confirm each of the following by checking the box.

Please fill out our survey

P.S. Have feedback/comments about our review process? Leave a comment here

Editor and Review Templates

The editor template can be found here.

The review template can be found here.

cmarmo commented 2 months ago

Hello @jsdodge ! Thank you for submitting THzTools to pyOpenSci. Sorry for the delay of my answer! I'm Chiara and I'm going to take care of your submission for the initial editorial checks. I will be back to you by the end of the week. Thanks for your patience!

jsdodge commented 2 months ago

Hi Chiara,

Thanks for the update on the timeline. Do you have any suggestions for ways that we can continue development without disrupting the review? Would it be ok, for example, to continue developing on the dev branch as long as we don't merge these into the main branch?

cmarmo commented 2 months ago

Do you have any suggestions for ways that we can continue development without disrupting the review? Would it be ok, for example, to continue developing on the dev branch as long as we don't merge these into the main branch?

I'm starting the editorial checks right now: I can perform them on the main branch if you prefer, once we agree on a version to be submitted you can tag the last modifications and we can update the description of the issue. Would that be ok for you?

cmarmo commented 2 months ago

Editor in Chief checks

Hi @jsdodge ! Thank you again for submitting your package for pyOpenSci review. Below are the basic checks that your package needs to pass to begin our review. If some of these are missing, we will ask you to work on them before the review process begins.

Please check our Python packaging guide for more information on the elements below.

Editor comments

THzTools is in excellent shape, congratulations! I have some minor comments before starting looking for an editor:

Do you have any suggestions for ways that we can continue development without disrupting the review? Would it be ok, for example, to continue developing on the dev branch as long as we don't merge these into the main branch?

I'm starting the editorial checks right now: I can perform them on the main branch if you prefer, once we agree on a version to be submitted you can tag the last modifications and we can update the description of the issue. Would that be ok for you?

I realize my answer was a bit out of scope... sorry for that. Indeed, I believe it is a good idea to continue the development in a separate branch during review: however, reviewers might ask for modifications too and everything would in principle end in a new version accepted at the end of the review process. Please just clarify with reviewers in which branch you are addressing their comments, we had some misunderstanding in the past.

jsdodge commented 2 months ago

Hi Chiara,

Thanks! We can add installation instructions to the README.md file and change the issue template right away. This raises a similar question to my earlier one: should we do this in the main branch? Normally we would also bump the version number when making changes in main, but the submission version is listed as v0.5.0.

Regarding the CONTRIBUTING.md file, we have a contributing.rst file in /docs/source/. Could you recommend a way to include this information at the top level in a CONTRIBUTING.md file and in the documentation without duplicating it? A related question is whether we can do the same thing with the CODE_OF_CONDUCT.md file. Currently we just have a GitHub link to CODE_OF_CONDUCT.md in contributing.rst, but it would be better to include it directly in the documentation. We used a link because we didn't know how to use ReST to pull it into the documentation source.

cmarmo commented 2 months ago

Thanks! We can add installation instructions to the README.md file and change the issue template right away. This raises a similar question to my earlier one: should we do this in the main branch? Normally we would also bump the version number when making changes in main, but the submission version is listed as v0.5.0.

Technically the review is not started yet: once done with the changes we can edit the issue description.

Regarding the CONTRIBUTING.md file, we have a contributing.rst file in /docs/source/. Could you recommend a way to include this information at the top level in a CONTRIBUTING.md file and in the documentation without duplicating it?

Some general information would be enough in the CONTRIBUTING.md file: a link to contributing.rst there will complete the instructions. See for example what is done in one of the previously accepted packages.

A related question is whether we can do the same thing with the CODE_OF_CONDUCT.md file. Currently we just have a GitHub link to CODE_OF_CONDUCT.md in contributing.rst, but it would be better to include it directly in the documentation. We used a link because we didn't know how to use ReST to pull it into the documentation source.

Your solution is fine with our standards: if you prefer to have the code of conduct in the documentation then you can use the same approach suggested for the CONTRIBUTING.md file and link the documentation reference inside.

jsdodge commented 2 months ago

I just released THzTools v0.5.1 with the changes that you requested. I've also updated the version number in the submission documentation.

cmarmo commented 2 months ago

Thank you @jsdodge ! Time for me to look for an editor!

jsdodge commented 1 month ago

Hi @cmarmo, I have just updated the submission with the latest release, v0.5.2.

Could you please update me on the editor search?

cmarmo commented 1 month ago

Thank you @jsdodge for the follow-up

Could you please update me on the editor search?

I'm sorry to say that I am still looking... I guess the end of summer plus the beginning of the academic year are not making things easier.... thanks for your patience!

lwasser commented 3 weeks ago

@cmarmo thank you so much for leading the pre-checks for this package!! @jsdodge we just have had a rotation for our EiC (we do this every 3 months). To help us get caught up, I was able to find an editor from our team to take on this package! @banesullivan !! The next step here is to find reviewers. Do you have any reviewers in mind that we could reach out to? This area if quite specific and we'd like to have atleast one person with domain expertise, involved in the review. many thanks for your patience!

jsdodge commented 3 weeks ago

Thanks @lwasser ! And hello @banesullivan . We look forward to working with you on the review.

Would it be possible for us to contact potential reviewers privately before suggesting them to you? I'm hesitant to list people publicly here without consulting them first.

lwasser commented 3 weeks ago

@jsdodge of course. I think reaching out to them privately is ideal. Normally we allow one suggestion from the author(s) and then we will try to find a second. The second reviewer can be more generally focused on packaging/usability. Finding reviewers has taken some time lately.

jsdodge commented 3 weeks ago

Hi @lwasser and @banesullivan , Romain Peretti (ORCID) has kindly agreed to help with the review. He leads the @THzbiophotonics group at CNRS in Lille, France. Please let me know if you need further help with the review.

lwasser commented 3 weeks ago

@jsdodge this is great. I'll leave a few notes and then will let @banesullivan step in. We may have a second reviewer. In the meantime, does Romain have a GitHub handle so we can add them to this issue? The review will happen fully in this issue with links to any issues or pr's opened of course!

jsdodge commented 3 weeks ago

Thank you, @lwasser , I'm glad that you may have found a second reviewer. I believe that Romain's GitHub handle is @THzbiophotonics, but I'll check.

jsdodge commented 3 weeks ago

Hello, please use the GitHub handle @Romain-Peretti for Romain. Thanks!

lwasser commented 3 weeks ago

ok fantastic. ✅ we have two reviewers now assigned to this issue. @frank1010111 and @Romain-Peretti I believe we are at this step in the review process and @banesullivan can take the review forward from here!! Bane can you please onboard the reviewers and kick things off?

Thank you all. Please let me know if i can be helpful moving forward!

banesullivan commented 3 weeks ago

:wave: Hi @frank1010111 and @Romain-Peretti! Thank you for volunteering to review THzTools for pyOpenSci! I'm excited to kick off this review and I'll try to chime in with a few comments of my own but overall THzTools is looking in great shape -- wonderful work, @jsdodge!

Please fill out our pre-review survey

Before beginning your review, please fill out our pre-review survey. This helps us improve all aspects of our review and better understand our community. No personal data will be shared from this survey - it will only be used in an aggregated format by our Executive Director to improve our processes and programs.

The following resources will help you complete your review:

  1. Here is the reviewers guide. This guide contains all of the steps and information needed to complete your review.
  2. Here is the review template that you will need to fill out and submit here as a comment, once your review is complete.

Please get in touch with any questions or concerns!

Your review is due: November ~8th, 2024

I've set the due date ~3 weeks out from now, but we're flexible here so please don't rush yourself if life or other things pop up and require you to delay a bit.

Reviewers: @frank1010111 @Romain-Peretti Due date: 2024-11-08

frank1010111 commented 3 weeks ago

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

Documentation

The package includes all the following forms of documentation:

Readme file requirements The package meets the readme requirements below:

The README should include, from top to bottom:

NOTE: If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the a badge for pyOpenSci peer-review will be provided upon acceptance.)

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider whether:

Functionality

For packages also submitting to JOSS

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

Final approval (post-review)

Estimated hours spent reviewing: 4

Review Comments

jsdodge commented 3 weeks ago

Hi @frank1010111 , thanks for your feedback so far. How would you like to proceed with the code updates? I've already addressed the issues that you posted, but just in my local dev branch. At what point would you recommend that I push these changes to the remote, merge them with the main branch, and create a new release? Should I wait for you to complete your review before doing any or all of these?

frank1010111 commented 3 weeks ago

I appreciate it. If you want me to review the dev branch, I'm okay doing that. Whatever works best for you is good for me.

jsdodge commented 3 weeks ago

In that case, I merged the changes into the main branch and created a new release, THzTools v0.5.3 (beta). I also closed the issues that you raised. I see that there are several unchecked items, so I'll wait for further recommendations.

banesullivan commented 2 weeks ago

Hey @jsdodge, I anticipate that a few more changes will be made during this review process, so there is no need to issue a new release for each of these changes during the review (unless you prefer to do so!). As the review finalizes though, we can make sure a release is issued and update the version specifier in the submission.

Moving forward, would you please try to link back to this GitHub issue by pasting the URL link in the descriptions of any Pull Requests or new Issues? This will help us track changes to the software that were a direct result of this review and help us check off items in the reviewers comments. Thanks!

jsdodge commented 2 weeks ago

Thanks for the suggestions, @banesullivan , will do.

frank1010111 commented 2 weeks ago

Today, I checked the paper draft created by a github action and found it meets JOSS specifications.

frank1010111 commented 2 weeks ago

I've got to say, your documentation is beautifully organized.

jsdodge commented 2 weeks ago

Thank you! It's nice to have that effort recognized.

frank1010111 commented 2 weeks ago

A nit for your documentation is that you've got docstrings for the parameters of your dataclasses, but Sphinx expects docstrings for your attributes. For instance, the lines here in NoiseModel are rendered fine, but Sphinx doesn't know to put them with these attributes.

The solution to this oddity is something like this

@dataclass
class Snake:
    """Oh no, it's a snake!"""

    width: float
    """Width of the snake."""

    length: float
    """Length of the snake."""

Once again, it's a nit that you don't have to change. Your documentation is perfectly readable as is. Just a fun fact for you if you're as obsessive about these things as me.

jsdodge commented 2 weeks ago

Thanks for that suggestion. Just to clarify, I'm using the numpydoc extension, so are you recommending that I repeat the class Parameters section in a separate Attributes section, as in the ExampleError class listed in the Sphinx Example NumPy Style Python Docstrings?

frank1010111 commented 2 weeks ago

I did not know about that example. That's amazing

frank1010111 commented 2 weeks ago

I've run and played around with the example notebooks, and I'm overall very pleased with the package and documentation. Very nice work. After that last issue is closed, my review ought to be complete and I can recommend acceptance.

Romain-Peretti commented 2 weeks ago

Sorry I did not begin yet

I ll try to do it from next Monday.

Romain

On 19/10/2024 17:22, Frank Male wrote:

I've run and played around with the example notebooks, and I'm overall very pleased with the package and documentation. Very nice work. After that last issue is closed, my review ought to be complete and I can recommend acceptance.

— Reply to this email directly, view it on GitHub https://github.com/pyOpenSci/software-submission/issues/209#issuecomment-2423962540, or unsubscribe https://github.com/notifications/unsubscribe-auth/BL6ZGXYHOADPEILNLRCTZ6TZ4J2LJAVCNFSM6AAAAABL2M3B32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRTHE3DENJUGA. You are receiving this because you were mentioned.Message ID: @.***>

--

Dr. Romain PERETTI (hab.) Leader of TeraHertz Biophotonics activity TeraHertz Photonics Group @ IEMN - CNRS (UMR 8520) Senior Member and Optics Express Associate editor @ OPTICA Phone: +33 32 019 78 76 @.***

--------------UBLZWRgfADBXzwtyHFh92bjE Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 8bit

<!DOCTYPE html>

Sorry I did not begin yet


I ll try to do it from next Monday.


Romain

On 19/10/2024 17:22, Frank Male wrote:

I've run and played around with the example notebooks, and I'm overall very pleased with the package and documentation. Very nice work. After that last issue is closed, my review ought to be complete and I can recommend acceptance.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: <pyOpenSci/software-submission/issues/209/2423962540@github.com>

-- 
Dr. Romain PERETTI (hab.)
Leader of TeraHertz Biophotonics activity
TeraHertz Photonics Group @ IEMN - CNRS (UMR 8520)
Senior Member and Optics Express Associate editor @ OPTICA
Phone: +33 32 019 78 76

--------------UBLZWRgfADBXzwtyHFh92bjE--

jsdodge commented 2 weeks ago

Thanks for your help, @frank1010111 ! I've made the changes that you requested and created a new release, THzTools v0.5.4 (beta), in anticipation of your review approval. I appreciate your suggestions and your promptness in completing the review.

jsdodge commented 2 weeks ago

Thanks, @Romain-Peretti , I look forward to your feedback. As noted in the comment above, I have issued a new release in response to several helpful suggestions from @frank1010111 . I've updated the submission to indicate the updated release.

Romain-Peretti commented 1 week ago

So I feel very the same way as @frank1010111 it is very impressive and I bet useful for quite a lot of people (at least a great playground for my next students to understand a bit all of that).

I have a question about the way the Hessian are computed. We struggled during month to find a good way to calculate the hessian in one of our project and I am not sure to follow how it is done here. It looks like you are computing the Hessian by yourself or am I wrong ?

Romain

jsdodge commented 1 week ago

Thank you for your feedback! I'm glad you think the software will be useful.

To your question, for the noisefit function we use scipy.minimize with the BFGS method to do the optimization, which does not require the Hessian. We tried other methods that did require the Hessian, however, so we have an internal function, _hess_noisefit to compute it that we left in the main code, in case we find a way to use it later to improve the computational speed. We compute the Jacobian explicitly in _jac_noisefit.

For the fit function, we use scipy.least_squares to do the minimization (see here), which also just uses the Jacobian. Internally, we fit for mu in addition to the fit parameters of the frequency response function, so the Jacobian that we compute internally is different from the Jacobian with respect to the fit parameters supplied to fit. We compute that Jacobian here.