[REVIEW]: libsoni: A Python Toolbox for Sonifying Music Annotations and Feature Representations

editorialbot commented 6 months ago

Submitting author: !--author-handle-->@yiitozer@osorensen<!--end-editor-- Reviewers: @expectopatronum, @ren-zeng Archive: 10.5281/zenodo.11085871

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/467e051d1f0e7161dd58acb3a85d6d27"><img src="https://joss.theoj.org/papers/467e051d1f0e7161dd58acb3a85d6d27/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/467e051d1f0e7161dd58acb3a85d6d27/status.svg)](https://joss.theoj.org/papers/467e051d1f0e7161dd58acb3a85d6d27)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@expectopatronum & @ren-zeng, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @osorensen know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @ren-zeng

📝 Checklist for @expectopatronum

editorialbot commented 6 months ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 6 months ago

Software report:

github.com/AlDanial/cloc v 1.90  T=0.09 s (1135.7 files/s, 378623.1 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
CSV                             28              0              0          18880
HTML                            21            891             60           4241
SVG                              1              0              0           2671
Python                          12            543            737            959
JavaScript                      12            131            213            871
CSS                              4            189             35            776
Jupyter Notebook                 5              0           1363            498
Markdown                         2             28              0            232
TeX                              1             14              0            133
YAML                             2              3              4             45
reStructuredText                11             20             46             33
DOS Batch                        1              8              1             26
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                           101           1831           2466          29374
-------------------------------------------------------------------------------

Commit count by author:

   145  leob
    78  yiitozer
    69  Leo Brütting
    22  Yigitcan Oezer
     4  Yigitcan Özer
     2  Øystein Sørensen

editorialbot commented 6 months ago

Paper file info:

📄 Wordcount for paper.md is 2366

✅ The paper includes a Statement of need section

editorialbot commented 6 months ago

License info:

🟡 License found: Other (Check here for OSI approval)

editorialbot commented 6 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7512227 is OK
- 10.21105/joss.03434 is OK
- 10.21105/joss.03326 is OK
- 10.1109/MSP.2018.2869928 is OK
- 10.1109/MSP.2018.2875349 is OK
- 10.1145/2964284.2973795 is OK
- 10.1007/978-3-319-21945-5 is OK
- 10.25080/Majora-7b98e3ed-003 is OK
- 10.5281/zenodo.1415016 is OK
- 10.5281/zenodo.1416032 is OK
- 10.5281/zenodo.849741 is OK
- 10.1145/1631272.1631459 is OK
- 10.5281/zenodo.1417145 is OK

MISSING DOIs

- No DOI given, and none found for title: Tempogram Toolbox: MATLAB Tempo and Pulse Analysis...
- No DOI given, and none found for title: Sonification Report: Status of the Field and Resea...

INVALID DOIs

- None

editorialbot commented 6 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

osorensen commented 5 months ago

👋 @expectopatronum, @ren-zeng, can you please update us on how it's going with your reviews?

ren-zeng commented 5 months ago

Review checklist for @ren-zeng

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/groupmm/libsoni?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@yiitozer) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

ren-zeng commented 5 months ago

👋 @expectopatronum, @ren-zeng, can you please update us on how it's going with your reviews?

Hi @osorensen, thanks for the reminder. I had some deadlines earlier and now I can mainly focus on this review. Sorry for the delay.

expectopatronum commented 5 months ago

👋 @expectopatronum, @ren-zeng, can you please update us on how it's going with your reviews?

Hi! Thanks for the reminder! I also had a deadline (submitted on Saturday) and was planning to work on the review this week!

expectopatronum commented 5 months ago

Review checklist for @expectopatronum

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/groupmm/libsoni?
[x] License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@yiitozer) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

ren-zeng commented 5 months ago

@yiitozer

[ ] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)
- The authors mentioned the following performance statement. However, there is no empirical support for improved efficiency.
- Secondly, we re-implemented and included all necessary components (e.g., sound generators based on sinusoidal models and click sounds), even though similar basic functionality is available in librosa and libfmp. By doing so, libsoni offers a coherent API along with convenient but easily modifiable parameter presets. Additionally, the implementations are more efficient than previous software.
[ ] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
- The API is satisfactory, it would be better if
  - there are minimal examples for each core function in the documentation.
  - all core functions specify the expected shape of the input numpy array (and what each dimension means) and the shape of the resulting array.
[ ] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
- There are tests but they cover very limited scenarios of the input data. It would be better if
- the authors use a more comprehensive testing framework like unittest and use randomly generated input instead of hand-coded minimal examples.
- the test input arrays are non-trivial and have more complex dimensions.
- the test covers more negative examples so that the user can know what properties the inputs need to satisfy.

yiitozer commented 5 months ago

Dear @ren-zeng, thank you very much for your constructive suggestions.

Performance: We will include a comparative analysis with librosa and libfmp in the Jupyter Notebooks.
Functionality documentation:
- We indeed provided minimal examples per core function in the Jupyter notebooks. Shall we also integrate them to the API?
- The explicit explanation of the dimension is a good point, we will add the shapes of the resulting arrays.
Automated tests:
- Since the toolbox is designed to sonify input annotations, we chose not to use randomly generated inputs to have a controlled scenario. Could you please elaborate on this suggestion?
- We will also add tests for the spectrogram case, where the input dimensions are more complex.
- We will add negative examples, thank you for your suggestion.

yiitozer commented 5 months ago

Dear @ren-zeng,

Performance: We have chosen to remove this statement, as the basic functions provided by libfmp and librosa show similar performances as those by libsoni: https://github.com/groupmm/libsoni/commit/859ea29843cd7a886f098e763baa8ef5e60c9458

Functionality documentation: We updated the API documentation indicating the explicit shapes of input and output arrays: https://groupmm.github.io/libsoni/build/html/index.html However, since we provided minimal examples of the core functions in the notebooks, adding those to the API documentation would result in redundant duplication of those examples.

Automated tests: We changed the tests to the comprehensive testing framework unittest and added further test cases, including negative examples. Since the toolbox is designed to sonify input annotations, we chose not to use randomly generated inputs to have a controlled scenario. Could you please elaborate on this suggestion?

ren-zeng commented 5 months ago

Dear @yiitozer,

Thank you for your update on the package and the paper. On my side, everything looks good now. Regarding my comment on the randomly generated inputs, I was mainly concerned, as a potential user, about what kinds of inputs are valid. But since you already added the required array shapes, it is not necessary anymore.

Hi @osorensen, I have checked all the boxes and my review is finished.

expectopatronum commented 5 months ago

I am deeply sorry for taking so long! I (once again) overestimated how much time I would have with a toddler at home 🙃

After solving a few issues directly by opening a Github issue, there are only a few things left:

Documentation: It took me a while to find the link to the documentation (in the sidebar of the github repo), maybe it could also be linked somewhere directly in the README?
For me it is not entirely clear whether by "Documentation" (in the review check list) the linked documentation is meant or the README + the linked documentation. In the first case I cannot check some of the boxes, because the installation instructions, example usage, ... is not described in what I think is the "documentation". Maybe it would be sufficient to mention in the starting page of the documentation that more details (installation instructions, ...) can be found in the Github repository. I'm thinking of the case where someone finds the package github.io page and does not know about the repository.
I did have the same question about the performance claims as @ren-zeng but this is solved now.
This is a very minor thing and I was not even sure if I should mention it but it bugs me a bit that both "Licence" and "License" are used.

Besides that, the package is great and I would have loved to have something like this when I started my PhD! I think it will be very useful for research in MIR and I think it will be easy to get started with it thanks to the example notebooks (which is not surprising coming from this group :)).

Thanks for your patience and best regards Verena

osorensen commented 5 months ago

Thanks a lot for completing your review @ren-zeng!

osorensen commented 5 months ago

Thanks a lot for your comments @expectopatronum. @yiitozer, please let us know here when the additional issues pointed out by @expectopatronum have been addressed.

yiitozer commented 5 months ago

Dear @expectopatronum,

* Documentation: It took me a while to find the [link to the documentation](https://groupmm.github.io/libsoni/build/html/index.html) (in the sidebar of the github repo), maybe it could also be linked somewhere directly in the README?

* For me it is not entirely clear whether by "Documentation" (in the review check list) the linked documentation is meant or the README + the linked documentation. In the first case I cannot check some of the boxes, because the installation instructions, example usage, ... is not described in what I think is the "documentation". Maybe it would be sufficient to mention in the starting page of the documentation that more details (installation instructions, ...) can be found in the Github repository. I'm thinking of the case where someone finds the package github.io page and does not know about the repository.

Thank you very much for your suggestions, we added a link to the documentation in the README file and a link to the GitHub repository in the API documentation, as well as renamed licence to license :) : https://github.com/groupmm/libsoni/pull/13

Besides that, the package is great and I would have loved to have something like this when I started my PhD! I think it will be very useful for research in MIR and I think it will be easy to get started with it thanks to the example notebooks (which is not surprising coming from this group :)).

Thanks for your patience and best regards Verena

Thank you very much for your constructive and motivating comment!! :) We're very happy to get such feedback!

Best, Yigit

yiitozer commented 5 months ago

Dear @osorensen, we're done with our changes based on the reviews!

osorensen commented 5 months ago

Thanks for responding so quickly, @yiitozer! @expectopatronum, could you please go through the changes, and update your review checklist if your points have been addressed?

expectopatronum commented 5 months ago

Dear @osorensen, everything looks great now, and all points are checked.

osorensen commented 5 months ago

@editorialbot generate pdf

osorensen commented 5 months ago

@editorialbot check references

editorialbot commented 5 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7512227 is OK
- 10.21105/joss.03434 is OK
- 10.21105/joss.03326 is OK
- 10.1109/MSP.2018.2869928 is OK
- 10.1109/MSP.2018.2875349 is OK
- 10.1145/2964284.2973795 is OK
- 10.1007/978-3-319-21945-5 is OK
- 10.25080/Majora-7b98e3ed-003 is OK
- 10.5281/zenodo.1415016 is OK
- 10.5281/zenodo.1416032 is OK
- 10.5281/zenodo.849741 is OK
- 10.1145/1631272.1631459 is OK
- 10.5281/zenodo.1417145 is OK

MISSING DOIs

- No DOI given, and none found for title: Tempogram Toolbox: MATLAB Tempo and Pulse Analysis...
- No DOI given, and none found for title: Sonification Report: Status of the Field and Resea...

INVALID DOIs

- None

editorialbot commented 5 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

osorensen commented 5 months ago

@yiitozer, I have now read through the final manuscript and have nothing to add or correct.

At this point could you:

[ ] Make a tagged release of your software, and list the version tag of the archived version here.
[ ] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
[ ] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[ ] Please list the DOI of the archived version here.

I can then move forward with recommending acceptance of the submission.

yiitozer commented 5 months ago

Dear @osorensen, thank you very much for the detailed instructions.

We have tagged the release of the software as v1.0.2 and published it on PyPI: https://pypi.org/project/libsoni/
Archived the reviewed software on Zenodo: https://zenodo.org/records/11085871
The DOI of the archived version: 10.5281/zenodo.11085871

osorensen commented 5 months ago

@editorialbot set v1.0.2 as version

editorialbot commented 5 months ago

Done! version is now v1.0.2

osorensen commented 5 months ago

@editorialbot set 10.5281/zenodo.11085871 as archive

editorialbot commented 5 months ago

Done! archive is now 10.5281/zenodo.11085871

osorensen commented 5 months ago

@editorialbot recommend-accept

editorialbot commented 5 months ago

Attempting dry run of processing paper acceptance...

editorialbot commented 5 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7512227 is OK
- 10.21105/joss.03434 is OK
- 10.21105/joss.03326 is OK
- 10.1109/MSP.2018.2869928 is OK
- 10.1109/MSP.2018.2875349 is OK
- 10.1145/2964284.2973795 is OK
- 10.1007/978-3-319-21945-5 is OK
- 10.25080/Majora-7b98e3ed-003 is OK
- 10.5281/zenodo.1415016 is OK
- 10.5281/zenodo.1416032 is OK
- 10.5281/zenodo.849741 is OK
- 10.1145/1631272.1631459 is OK
- 10.5281/zenodo.1417145 is OK

MISSING DOIs

- No DOI given, and none found for title: Tempogram Toolbox: MATLAB Tempo and Pulse Analysis...
- No DOI given, and none found for title: Sonification Report: Status of the Field and Resea...

INVALID DOIs

- None

editorialbot commented 5 months ago

:wave: @openjournals/sbcs-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/5286, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

yiitozer commented 5 months ago

Dear @osorensen, is the paper than acceptable with the command "editorialbot accept"? :)

osorensen commented 5 months ago

No, you'll have to wait for the editor-in-chief to make the final decision :)

yiitozer commented 5 months ago

Alright, thank you for your quick response! @osorensen

oliviaguest commented 5 months ago

@editorialbot recommend-accept

editorialbot commented 5 months ago

Attempting dry run of processing paper acceptance...

editorialbot commented 5 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7512227 is OK
- 10.21105/joss.03434 is OK
- 10.21105/joss.03326 is OK
- 10.1109/MSP.2018.2869928 is OK
- 10.1109/MSP.2018.2875349 is OK
- 10.1145/2964284.2973795 is OK
- 10.1007/978-3-319-21945-5 is OK
- 10.25080/Majora-7b98e3ed-003 is OK
- 10.5281/zenodo.1415016 is OK
- 10.5281/zenodo.1416032 is OK
- 10.5281/zenodo.849741 is OK
- 10.1145/1631272.1631459 is OK
- 10.5281/zenodo.1417145 is OK

MISSING DOIs

- No DOI given, and none found for title: Tempogram Toolbox: MATLAB Tempo and Pulse Analysis...
- No DOI given, and none found for title: Sonification Report: Status of the Field and Resea...

INVALID DOIs

- None

editorialbot commented 5 months ago

:wave: @openjournals/sbcs-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/5305, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

oliviaguest commented 5 months ago

@yiitozer please review the PR (https://github.com/groupmm/libsoni/pull/14) and then recompile the paper and let me know if this is a version you are happy to publish. ☺️

yiitozer commented 4 months ago

Dear @oliviaguest, thank you very much, I have merged the pull request. I would be more than happy to publish this version :smiley:

oliviaguest commented 4 months ago

@editorialbot recommend-accept

editorialbot commented 4 months ago

Attempting dry run of processing paper acceptance...

editorialbot commented 4 months ago

:wave: @openjournals/sbcs-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/5310, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

editorialbot commented 4 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.5281/zenodo.7512227 is OK
- 10.21105/joss.03434 is OK
- 10.21105/joss.03326 is OK
- 10.1109/MSP.2018.2869928 is OK
- 10.1109/MSP.2018.2875349 is OK
- 10.1145/2964284.2973795 is OK
- 10.1007/978-3-319-21945-5 is OK
- 10.25080/Majora-7b98e3ed-003 is OK
- 10.5281/zenodo.1415016 is OK
- 10.5281/zenodo.1416032 is OK
- 10.5281/zenodo.849741 is OK
- 10.1145/1631272.1631459 is OK
- 10.5281/zenodo.1417145 is OK

MISSING DOIs

- No DOI given, and none found for title: Tempogram Toolbox: MATLAB Tempo and Pulse Analysis...
- No DOI given, and none found for title: Sonification Report: Status of the Field and Resea...

INVALID DOIs

- None

oliviaguest commented 4 months ago

@editorialbot generate pdf

openjournals / joss-reviews