search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
722 stars 38 forks source link

[REVIEW]: onsetsync: An R Package for Onset Synchrony Analysis #5395

Closed editorialbot closed 10 months ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@tuomaseerola<!--end-author-handle-- (Tuomas Eerola) Repository: https://github.com/tuomaseerola/onsetsync Branch with paper.md (empty if default branch): master Version: 0.5.1 Editor: !--editor-->@faroit<!--end-editor-- Reviewers: @pmcharrison, @mrocamora Archive: 10.5281/zenodo.10050346

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e"><img src="https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e/status.svg)](https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@pmcharrison & @mrocamora, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @faroit know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @mrocamora

📝 Checklist for @pmcharrison

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf
editorialbot commented 1 year ago 
Software report:

github.com/AlDanial/cloc v 1.88  T=0.12 s (886.6 files/s, 223445.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                       4           2099           1928           7019
HTML                            46           1640            146           6789
XML                              6              0            129           1929
R                               29            203            621           1097
Jupyter Notebook                 1              0            104            500
Markdown                         5            202              0            500
Rmd                              8            335            570            480
TeX                              1             34              4            297
YAML                             3              1              4             71
SVG                              1              0              1             11
CSS                              1              0              0              1
JSON                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                           106           4514           3507          18695
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository
editorialbot commented 1 year ago

Wordcount for paper.md is 4834

editorialbot commented 1 year ago

Failed to discover a Statement of need section in paper

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

editorialbot commented 1 year ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.17605/OSF.IO/SFXA2 is OK
- 10.17605/OSF.IO/KS325 is OK

MISSING DOIs

- 10.1162/comj_a_00002 may be a valid DOI for title: Investigations of between-hand synchronization in Magaloff’s Chopin
- 10.18061/1811/51203 may be a valid DOI for title: Inter-group entrainment in Afro-Brazilian Congado ritual
- 10.1109/icassp.2014.6854953 may be a valid DOI for title: Improved musical onset detection with convolutional neural networks
- 10.3389/fpsyg.2018.02232 may be a valid DOI for title: Analyzing multivariate dynamics using cross-recurrence quantification analysis (CRQA), diagonal-cross-recurrence profiles (dcrp), and multidimensional recurrence quantification analysis (MDRQA)–a tutorial in R
- 10.1177/0305735614555790 may be a valid DOI for title: Timing deviations in jazz performance: The relationships of selected musical variables on horizontal and vertical timing relations: A case study
- 10.1109/lsp.2020.3045915 may be a valid DOI for title: Adversarial unsupervised domain adaptation for harmonic-percussive source separation
- 10.1145/2660168.2660182 may be a valid DOI for title: Creating corpora for computational research in Arab-Andalusian music
- 10.1109/icassp.2016.7471640 may be a valid DOI for title: A generalized bayesian model for tracking long metrical cycles in acoustic music signals
- 10.1098/rsif.2013.1125 may be a valid DOI for title: Optimal feedback correction in string quartet synchronization
- 10.3758/s13423-012-0371-2 may be a valid DOI for title: Sensorimotor synchronization: a review of recent research (2006–2012)
- 10.3758/bf03206433 may be a valid DOI for title: Sensorimotor synchronization: A review of the tapping literature
- 10.3389/fnins.2016.00285 may be a valid DOI for title: Both isochronous and non-isochronous metrical subdivision afford precise and stable ensemble entrainment: A corpus study of Malian jembe drumming
- 10.1016/j.humov.2008.02.016 may be a valid DOI for title: Sensorimotor synchronization with adaptively timed sequences

INVALID DOIs

- http://dx.doi.org/10.1098/rsos.171520 is INVALID because of 'https://doi.org/' prefix
- https://doi.org/10.1177/1029864919844809 is INVALID because of 'https://doi.org/' prefix
- http://dx.doi.org/10.18061/emr.v16i1.7555 is INVALID because of 'https://doi.org/' prefix
- http://dx.doi.org/10.18061/emr.v16i1.7555 is INVALID because of 'https://doi.org/' prefix
faroit commented 1 year ago

👋🏼 @tuomaseerola @pmcharrison, @mrocamora this is the review thread for the paper. All of our communications will happen here from now on.

As a reviewer, the first step is to create a checklist for your review by entering

@editorialbot generate my checklist

as the top of a new comment in this thread.

These checklists contain the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. The first comment in this thread also contains links to the JOSS reviewer guidelines.

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#5395 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if any of you require some more time. We can also use EditorialBot (our bot) to set automatic reminders if you know you'll be away for a known period of time.

Please feel free to ping me (@faroit) if you have any questions/concerns.

mrocamora commented 1 year ago

Review checklist for @mrocamora

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

pmcharrison commented 1 year ago

Review checklist for @pmcharrison

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

pmcharrison commented 1 year ago

Typos in this sentence:

These collection contain tens of hours annotated and verified

pmcharrison commented 1 year ago

Typo here:

a processes occurring within 100-2000 ms timescales

pmcharrison commented 1 year ago

Wrong citation format here:

Some datasets such as Chopin performances compiled by (Goebl et al., 2010)

pmcharrison commented 1 year ago

Wrong citation format here:

commonly been studied with tapping experiments Repp & Su (2013)

pmcharrison commented 1 year ago

Wrong citation format here:

A number of different measures of synchrony have been proposed. We follow the definitions 54 from (Clayton et al., 2020, p. 161)

pmcharrison commented 1 year ago

Eq. 1: sum symbol should say i

pmcharrison commented 1 year ago

Eq. 3: why not use d_i abbreviation here?

pmcharrison commented 1 year ago

Eq. 3: T has not been defined yet

pmcharrison commented 1 year ago

Eq. 4: why has T now been changed to n? Same in Eq. 5

pmcharrison commented 1 year ago

Eq. 4: i=i should be i=1, same in Eq. 5

pmcharrison commented 1 year ago

Eq. 5: isn't this the same as d bar, used in Eq. 2? If so, use the same symbol for both?

pmcharrison commented 1 year ago

Eq. 6: shouldn't the intro to this equation say onset time v instead of onset time w?

pmcharrison commented 1 year ago

Eq. 8: i=i should be i=1.

I'm also not sure what the subscript r means here.

pmcharrison commented 1 year ago

which is usually the relevant approach in music sense.

doesn't seem like idiomatic English.

pmcharrison commented 1 year ago

Typo here:

In order to be able to build the carry out such comparisons

pmcharrison commented 1 year ago

Wrong citation formats:

see (Schlüter & Böck, 2014) and (Böck et al., 2016), and for a full workflow of how to combine onset detection and annotation of the musical information, see (Clayton, Tarsitani, et al., 2021).

pmcharrison commented 1 year ago

three first are meta-data

grammatical typo, but also I think the correct number is two

pmcharrison commented 1 year ago

Table 2: it seems unfortunate that SD is used as a column name, because most readers will think this denotes standard deviation. Probably it's too late to change now though

pmcharrison commented 1 year ago

Typo:

As a broad overview, we can use the function plot_by_beat visualise the asynchrony relative to

pmcharrison commented 1 year ago

For example, in many genres different instruments contribute to specific subdivision of the beat.

subdivision should be plural.

pmcharrison commented 1 year ago

Need to be consistent about whether you write [a]synchronization or [a]synchronisation

pmcharrison commented 1 year ago

'granger' should be capitalised

pmcharrison commented 1 year ago

Regarding this checklist point,

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

I could not see any automated tests, but maybe I am missing them?

pmcharrison commented 1 year ago

Regarding this checklist point,

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

I could not see any, but maybe I am missing them?

pmcharrison commented 1 year ago

Regarding this checklist point,

State of the field: Do the authors describe how this software compares to other commonly-used packages?

I could not see any comparison, but maybe I missed it.

pmcharrison commented 1 year ago

There is a typo in this reference:

Müller m, Wiering f, Editors.

and this one:

A generalized bayesian

pmcharrison commented 1 year ago

@tuomaseerola I have gone through the paper and it generally looks good, I think the functionality is appealing and I particularly like the visualisation side. I can see that the pipeline would work very well for future researchers in that the input data format is quite general and once you have in that format it's very easy to apply your code to it.

The paper is a bit long for JOSS format but I think it works to have those detailed examples (I didn't check whether it exceeds the 1,000 word limit, it's probably close). There were quite a few typographical issues and I think the draft could do with some detailed proofing to complement what I was able to spot. I have made detailed individual comments above.

mrocamora commented 1 year ago

Regarding COIs, I declare that I participated in the Interpersonal Entrainment in Music Performance (IEMP) project, which took place from 2016 to 2018. I worked on preparing Uruguayan candombe data for the IEMP collection in collaboration with Luis Jure, Simone Tarsitani, and Martin Clayton. I also discussed and explored approaches for precise onset extraction from audio recordings with Martin Clayton and Tuomas Eerola. However, I think I can make an impartial assessment of the work presented to JOSS, whose focus is the analysis of onsets to assess musical synchrony.

mrocamora commented 1 year ago

Wrong citation format here:

with tapping experiments Repp & Su (2013)

mrocamora commented 1 year ago

Here we first define the onset time differences, 𝑑, ...

Then, in the definition of the precision measures, the term "asynchronies" (or "asynchrony values") is used for d, instead of "onset time differences". So I suggest: "Here we first define the onset time differences (or asynchronies), 𝑑, ..."

mrocamora commented 1 year ago

Eq. 1: Add summation limits.

mrocamora commented 1 year ago

Eq. 3: According to (Clayton et al., 2020, p. 161), its "the RMS of the pair-wise measurements" of asynchronies. So I guess it should be $I{1,i} - I{2,i}$ or just $d_i$. Besides, what $T$ stands for should be clarified.

mrocamora commented 1 year ago

Eq. 4 and 5: Summation starts at $i=1$ Eq.4: I suggest using $|\bar{s}|$ instead of $|\hat{s}|$ to be consistent with the use of $\bar{d}$ in Eq. 2.

mrocamora commented 1 year ago

Eq. 5: Seems that $\bar{d}$ should be used instead of $\hat{s_p}$ (as in Eq. 2). I don't see where the $p$ comes from (pairwise? if this is the case, note that it is not used in Eq. 2).

mrocamora commented 1 year ago

Mean relative asynchrony : mean position of an instrument’s onsets relative to average position $(\hat{\omega})$ of the group, ...

The position of an instrument's onsets was previously notated as $I$ ($I{1,i}$ and $I{2,i}$ in Eq. 1). I suggest using a consistent notation for onsets. Besides, I also suggest using $\bar{w}$ instead of $\hat{w}$, as previously done in Eq. 2.

mrocamora commented 1 year ago

and the relative onset time $w$ is

Should be $v$ instead of $w$.

mrocamora commented 1 year ago

Eq. 7: It seems that $n$ should be $i$.

mrocamora commented 1 year ago

Eq. 8: I suggest using $\bar{s_r}$ instead of $\hat{s_r}$.

mrocamora commented 1 year ago

which is usually the relevant approach in music sense.

This assertion could be backed up by citing some relevant references.

mrocamora commented 1 year ago

In order to be able to build the carry out such comparisons, ...

There is a typo in this sentence, and it is not clear to what comparisons it refers to (see previous sentence).

mrocamora commented 1 year ago

(e.g. Librosa, Essentia, or MIR toolbox for Matlab, or Sonic Visualiser using well-known onset detection algorithms.

There is an extra "or" and the closing parenthesis is missing.