[REVIEW]: onsetsync: An R Package for Onset Synchrony Analysis

[editorialbot]

Submitting author: !--author-handle-->@tuomaseerola<!--end-author-handle-- (Tuomas Eerola) Repository: https://github.com/tuomaseerola/onsetsync Branch with paper.md (empty if default branch): master Version: 0.5.1 Editor: !--editor-->@faroit<!--end-editor-- Reviewers: @pmcharrison, @mrocamora Archive: 10.5281/zenodo.10050346

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e"><img src="https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e/status.svg)](https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@pmcharrison & @mrocamora, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @faroit know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @mrocamora

📝 Checklist for @pmcharrison

[mrocamora]

see (Schlüter & Böck, 2014) and (Böck et al., 2016), and for a full workflow of how to combine onset detection and annotation of the musical information, see (Clayton, Tarsitani, et al., 2021).

Wrong citation format.

[mrocamora]

(with precision of 6 digits, ...

Closing parenthesis is missing.

[mrocamora]

Table 2: It is noteworthy that the first row of the table has no onset for the selected instruments. This may be confusing and could be explicitly clarified.

I wonder if adding a comment after the following line could work.

Information about cycles and beat subdivisions is generally based on manual annotation, and originates separately from the automatically extracted onsets; the two are combined to produce tables with onsets assigned to beat positions.

[mrocamora]

Sec 1.6 Onset summaries

Note that you need to load library(dplyr) to run the example code, as it internally uses the ungroup function. This is clearly specified in the Usage section in the repository but not in the paper. I suggest adding a Usage section to the paper. I am aware of the "Analysis Example" vignette, which has all the code needed to run the examples. However, the code to load the libraries is small enough to be included in the paper and could allow someone reading the paper to follow it by copy-pasting the code in R.

[mrocamora]

Table 3. : It seems that SD here is different from that of Table 2.

[mrocamora]

As a broad overview, we can use the function plot_by_beat visualise the asynchrony

to visualise

[mrocamora]

and towards the end of the piece tres ...

the tres

[mrocamora]

Overall this suggests that these two instruments ... although guitar is tightly aligned ... are not evident in tres ...

the guitar, the tres

[mrocamora]

Here we continue the analysis guitar and tres and calculate the asynchrony between them.

the analysis with the guitar and the tres

[mrocamora]

shows that bass is ahead (ie. -19 ms)

i.e.

[mrocamora]

We constrain the frequency a range of

Typo: to a range

[mrocamora]

we will plot the estimated frequency curves (Figure 6).

we plot the estimated ...

[mrocamora]

Regarding the review checklist, I could not find automated tests.

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

[mrocamora]

Regarding the review checklist, I could not find community guidelines.

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

[mrocamora]

Regarding the review checklist, I could not find a comparison to other commonly-used packages. This is probably because there are no other packages of this kind.

State of the field: Do the authors describe how this software compares to other commonly-used packages?

[mrocamora]

@tuomaseerola I went through the paper, the repo, and the web and checked the examples' functionality. It is an excellent contribution with relevance in several research scenarios. The word count for the paper.md file is 4834 (according to the editorial bot), surpassing the word limit, so we should check with @faroit on how to proceed. I have noted some typos and minor errors that are easy to fix and are listed above. Besides, some of the points in the review checklist are missing. Community guidelines should be added. As for the rest, automated tests and comparison to other packages, we should confirm with @faroit if they are mandatory.

[tuomaseerola]

Eq. 1: sum symbol should say i

Is this about Eq. 2 which has sum symbol?

[faroit]

@mrocamora @pmcharrison thank you already for your very detailed reviews. I value that you also went deeper into the notation of the paper. However, to free this issue up from too many comments, I suggest to do future discussion about notation in distinct issues in the submission repository. You can include in your review links to any new issues that you the reviewer believe to be impeding the acceptance of the repository.

[faroit]

@mrocamora

The word count for the paper.md file is 4834 (according to the editorial bot), surpassing the word limit.

even if most of the word count is driven by the tables, i would agree that the paper is too long and should be shorten by at least 1-2 pages.

please open an issue to track this.

Community guidelines should be added.

Yes, please open an issue.

As for the rest, automated tests and comparison to other packages, we should confirm with @faroit if they are mandatory.

automated test are preferred but not absolutely mandatory if step-by-step instructions are provided. In this case however, I would suggest to add a minimal set of automation (e.g. package installation, running)

I think a comparison (and references) to other packages are highly suggested, at least when it comes to general onset detection and evaluation.

[mrocamora]

@faroit @tuomaseerola @pmcharrison I've added an issue regarding word limit

https://github.com/tuomaseerola/onsetsync/issues/3#issue-1709098619

[mrocamora]

@faroit @tuomaseerola @pmcharrison I've added and issue about the community guidelines

https://github.com/tuomaseerola/onsetsync/issues/4#issue-1709099132

[mrocamora]

@faroit @tuomaseerola @pmcharrison I've added and issue to discuss the automation (test, install, etc.)

https://github.com/tuomaseerola/onsetsync/issues/5#issue-1709099745

[tuomaseerola]

Typo here:

a processes occurring within 100-2000 ms timescales

fixed

[tuomaseerola]

Typos in this sentence:

These collection contain tens of hours annotated and verified

fixed

[tuomaseerola]

Wrong citation format here:

Some datasets such as Chopin performances compiled by (Goebl et al., 2010)

fixed

[tuomaseerola]

fixed

[tuomaseerola]

Wrong citation format here:

A number of different measures of synchrony have been proposed. We follow the definitions 54 from (Clayton et al., 2020, p. 161)

fixed (and Rasch references altered as well).

[tuomaseerola]

Eq. 1: sum symbol should say i

i=1 added to the sum of the Eq. 2

[tuomaseerola]

Eq. 3: why not use d_i abbreviation here?

replaced the clunky notation with d_i

[tuomaseerola]

Eq. 3: T has not been defined yet

Switched to n to be consistent and defined it as "where n is the number of onset time differences. "

[tuomaseerola]

Eq. 4: i=i should be i=1, same in Eq. 5

fixed

[tuomaseerola]

Eq. 5: isn't this the same as d bar, used in Eq. 2? If so, use the same symbol for both?

fixed

[tuomaseerola]

Eq. 6: shouldn't the intro to this equation say onset time v instead of onset time w?

Yes, fixed, and all $\hat{w}$ symbols are now $\bar{w}$ symbols for consistency.

[tuomaseerola]

Eq. 8: i=i should be i=1.

I'm also not sure what the subscript r means here.

fixed i=1, removed r as subscript, and switched $\hat{s}$ to $\bar{s}$ to refer to mean.

[tuomaseerola]

doesn't seem like idiomatic English.

fixed (sentence eliminated to shorten the text)

[tuomaseerola]

In order to be able to

fixed

[tuomaseerola]

see (Schlüter & Böck, 2014) and (Böck et al., 2016), and for a full workflow of how to combine onset detection and annotation of the musical information, see (Clayton, Tarsitani, et al., 2021).

fixed

[tuomaseerola]

three first are meta-data

grammatical typo, but also I think the correct number is two

fixed

[tuomaseerola]

Table 2: it seems unfortunate that SD is used as a column name, because most readers will think this denotes standard ?> deviation. Probably it's too late to change now though

Switching from SD to Beat would make this variable clearer. It requires extensive revision, so will first estimate the amount of work needed.

[tuomaseerola]

Typo:

As a broad overview, we can use the function plot_by_beat visualise the asynchrony relative to

fixed

[tuomaseerola]

For example, in many genres different instruments contribute to specific subdivision of the beat.

subdivision should be plural.

section eliminated

[tuomaseerola]

Need to be consistent about whether you write [a]synchronization or [a]synchronisation

fixed (spelling with z).

[tuomaseerola]

'granger' should be capitalised

fixed

[tuomaseerola]

Regarding this checklist point,

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

I could not see any automated tests, but maybe I am missing them?

The first three code blocks will do this and the Documentation Quick Guide starts with this and the primary vignette ('onsetsync') will also have the basic steps coded in with build-in data.

[tuomaseerola]

Regarding this checklist point,

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

I could not see any, but maybe I am missing them?

These have now been added as a separate file CONTRIBUTING.md and linked from the landing page and documentation.

[tuomaseerola]

Regarding this checklist point,

State of the field: Do the authors describe how this software compares to other commonly-used packages?

I could not see any comparison, but maybe I missed it.

There is a Python library SyncPy that could provide some overlapping functionality but since it is not commonly used and it has not been updated since 2017, flagging this up felt unnecessary.

[tuomaseerola]

There is a typo in this reference:

Müller m, Wiering f, Editors.

and this one:

A generalized bayesian

fixed both (and some other minor bibtex quirks)

[tuomaseerola]

@tuomaseerola I have gone through the paper and it generally looks good, I think the functionality is appealing and I particularly like the visualisation side. I can see that the pipeline would work very well for future researchers in that the input data format is quite general and once you have in that format it's very easy to apply your code to it.

The paper is a bit long for JOSS format but I think it works to have those detailed examples (I didn't check whether it exceeds the 1,000 word limit, it's probably close). There were quite a few typographical issues and I think the draft could do with some detailed proofing to complement what I was able to spot. I have made detailed individual comments above.

Thanks for the perceptive comments and constructive suggestions. The length is now half of what it used to be as we chopped off sections to Documentation site and trimmed the text in other ways.

[tuomaseerola]

Wrong citation format here:

with tapping experiments Repp & Su (2013)

fixed

[tuomaseerola]

Here we first define the onset time differences, 𝑑, ...

Then, in the definition of the precision measures, the term "asynchronies" (or "asynchrony values") is used for d, instead of "onset time differences". So I suggest: "Here we first define the onset time differences (or asynchronies), 𝑑, ..."

thanks, this has been adopted. The terminology needs tightening up.