openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[REVIEW]: onsetsync: An R Package for Onset Synchrony Analysis #5395

Closed editorialbot closed 9 months ago

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@tuomaseerola<!--end-author-handle-- (Tuomas Eerola) Repository: https://github.com/tuomaseerola/onsetsync Branch with paper.md (empty if default branch): master Version: 0.5.1 Editor: !--editor-->@faroit<!--end-editor-- Reviewers: @pmcharrison, @mrocamora Archive: 10.5281/zenodo.10050346

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e"><img src="https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e/status.svg)](https://joss.theoj.org/papers/8d123fec2e6c1c33189dca188f10fa3e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@pmcharrison & @mrocamora, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @faroit know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Checklists

📝 Checklist for @mrocamora

📝 Checklist for @pmcharrison

tuomaseerola commented 1 year ago

Eq. 3: According to (Clayton et al., 2020, p. 161), its "the RMS of the pair-wise measurements" of asynchronies. So I guess it should be I1,i−I2,i or just di. Besides, what T stands for should be clarified.

Both issues fixed ($d_i$ and replace T with n.

tuomaseerola commented 1 year ago

Eq. 4 and 5: Summation starts at i=1 Eq.4: I suggest using |s¯| instead of |s^| to be consistent with the use of d¯ in Eq. 2.

fixed

tuomaseerola commented 1 year ago

Eq. 5: Seems that d¯ should be used instead of sp^ (as in Eq. 2). I don't see where the p comes from (pairwise? if this is the case, note that it is not used in Eq. 2).

fixed

tuomaseerola commented 1 year ago

Mean relative asynchrony : mean position of an instrument’s onsets relative to average position (ω^) of the group, ...

The position of an instrument's onsets was previously notated as I (I1,i and I2,i in Eq. 1). I suggest using a consistent notation for onsets. Besides, I also suggest using w¯ instead of w^, as previously done in Eq. 2.

These are great suggestions, these equations had inconsistencies in them and now $\bar{I}$ instead of $\hat{w}$.

tuomaseerola commented 1 year ago

and the relative onset time w is

Should be v instead of w.

fixed

tuomaseerola commented 1 year ago

Eq. 7: It seems that n should be i.

fixed

tuomaseerola commented 1 year ago

Eq. 8: I suggest using sr¯ instead of sr^.

fixed

tuomaseerola commented 1 year ago

relevant

fixed (this sentence no longer exists in the trimmed document)

tuomaseerola commented 1 year ago

In order to be able to build the carry out such comparisons, ...

There is a typo in this sentence, and it is not clear to what comparisons it refers to (see previous sentence).

fixed

tuomaseerola commented 1 year ago

(e.g. Librosa, Essentia, or MIR toolbox for Matlab, or Sonic Visualiser using well-known onset detection algorithms.

There is an extra "or" and the closing parenthesis is missing.

both issues fixed, thank you.

tuomaseerola commented 1 year ago

see (Schlüter & Böck, 2014) and (Böck et al., 2016), and for a full workflow of how to combine onset detection and annotation of the musical information, see (Clayton, Tarsitani, et al., 2021).

Wrong citation format.

fixed

tuomaseerola commented 1 year ago

(with precision of 6 digits, ...

Closing parenthesis is missing.

fixed

tuomaseerola commented 1 year ago

Information about cycles and beat subdivisions is generally based on manual annotation, and originates separately from the automatically extracted onsets; the two are combined to produce tables with onsets assigned to beat positions.

Good, a clarification has been added:

This explains why the first row of the table has no onsets but it contains annotation information (Cycle 1 and beat sub-division 1)

tuomaseerola commented 1 year ago

Sec 1.6 Onset summaries

Note that you need to load library(dplyr) to run the example code, as it internally uses the ungroup function. This is clearly specified in the Usage section in the repository but not in the paper. I suggest adding a Usage section to the paper. I am aware of the "Analysis Example" vignette, which has all the code needed to run the examples. However, the code to load the libraries is small enough to be included in the paper and could allow someone reading the paper to follow it by copy-pasting the code in R.

We agree with this and adding the explicit call to the library is short price to pay for the easy of use.

tuomaseerola commented 1 year ago

Table 3. : It seems that SD here is different from that of Table 2.

This table has now been omitted to save space but the point this stands; SD is the common abbreviation of standard deviation and unfortunately seems to create confusion. We will review during the revision whether switching from the sub-division (SD) to something else is the best solution.

tuomaseerola commented 1 year ago

As a broad overview, we can use the function plot_by_beat visualise the asynchrony

to visualise

fixed

tuomaseerola commented 1 year ago

and towards the end of the piece tres ...

the tres

fixed

tuomaseerola commented 1 year ago

Overall this suggests that these two instruments ... although guitar is tightly aligned ... are not evident in tres ...

the guitar, the tres

fixed

tuomaseerola commented 1 year ago

Here we continue the analysis guitar and tres and calculate the asynchrony between them.

the analysis with the guitar and the tres

This section is no longer in the article but has been fixed in the Documentation (articles)

tuomaseerola commented 1 year ago

shows that bass is ahead (ie. -19 ms)

i.e.

fixed

tuomaseerola commented 1 year ago

We constrain the frequency a range of

Typo: to a range

This section is no longer in the article but has been fixed in the Documentation (articles)

tuomaseerola commented 1 year ago

we will plot the estimated frequency curves (Figure 6).

we plot the estimated ...

This section is no longer in the article but has been fixed in the Documentation (articles)

tuomaseerola commented 1 year ago

Regarding the review checklist, I could not find automated tests.

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

This has been addressed in the documentation and also the three first code blocks in the manuscript function as a test of the functionality of the software using the build-in data.

tuomaseerola commented 1 year ago

Regarding the review checklist, I could not find community guidelines.

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

These have now been added as a separate file CONTRIBUTING.md and linked from the landing page and documentation.

tuomaseerola commented 1 year ago

Regarding the review checklist, I could not find a comparison to other commonly-used packages. This is probably because there are no other packages of this kind.

State of the field: Do the authors describe how this software compares to other commonly-used packages?

True, nothing too directly similar. There is a Python library SyncPy that could provide some overlapping functionality but since it is not commonly used and it has not been updated since 2017, flagging it up felt unnecessary.

tuomaseerola commented 1 year ago

@tuomaseerola I went through the paper, the repo, and the web and checked the examples' functionality. It is an excellent contribution with relevance in several research scenarios. The word count for the paper.md file is 4834 (according to the editorial bot), surpassing the word limit, so we should check with @faroit on how to proceed. I have noted some typos and minor errors that are easy to fix and are listed above. Besides, some of the points in the review checklist are missing. Community guidelines should be added. As for the rest, automated tests and comparison to other packages, we should confirm with @faroit if they are mandatory.

Thank you for the diligent and careful reading of the manuscript and that you explored the repo and tested the functionality of the commands. These were really constructive comments and we have incorporated them all as best as we could. We hope radical shortening and removing several sections into online Documentation will make the issue about the length less problematic for the journal.

pmcharrison commented 1 year ago

Thanks @tuomaseerola -- is your next draft complete, then? Shall we regenerate the paper and have another read?

tuomaseerola commented 1 year ago

Yes, @mrocamora and @pmcharrison. All revisions (including paper, unit tests, additional articles for documentation that were discarded from the paper) are in R1 branch, which I have not yet merged with the main.

faroit commented 1 year ago

@tuomaseerola can you update us on the status of changes, requested by the reviewers, please?

tuomaseerola commented 1 year ago

Sure, @faroit, I thought I did this on May 29 already, so all revisions (including the corrections requested to the paper, and unit tests, and additional articles for documentation that were discarded from the paper) have been completed the revisions (R1 branch) branch, although I have not merged these with the main branch as I assumed you would be able to see the branch. Should I merge R1 branch with the main branch?

faroit commented 1 year ago

Sure, @faroit, I thought I did this on May 29 already, so all revisions (including the corrections requested to the paper, and unit tests, and additional articles for documentation that were discarded from the paper) have been completed the revisions (R1 branch) branch, although I have not merged these with the main branch as I assumed you would be able to see the branch. Should I merge R1 branch with the main branch?

@tuomaseerola No, that's fine. I just wasn't just if this meant that the reviews were addresses in there. Merging can be done at the end of the review process before a new tag will be issued.

faroit commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

faroit commented 1 year ago

@mrocamora @pmcharrison can you please update us on the status of the review given the changes that @tuomaseerola has done?

As a large number of requested changes and responses were carried out in this thread it would be nice if both authors and reviewers can add 👍 to the comments above to mark that issues are resolved.

pmcharrison commented 1 year ago

Eq. 1: sum symbol should say i

Is this about Eq. 2 which has sum symbol?

Yes, sorry

pmcharrison commented 1 year ago

Eq. 3: T has not been defined yet

Switched to n to be consistent and defined it as "where n is the number of onset time differences. "

I can't see this change in the latest copy?

pmcharrison commented 1 year ago

@tuomaseerola I have started going through checking whether the responses have been made to my comments but it seems like quite a few already didn't make it into the revised manuscript, though it seems like you read them and gave thumbs up. Am I looking at the wrong version, or making some other mistake? I'm thinking primarily of small typographical things.

faroit commented 1 year ago

@tuomaseerola can you update us on the status of the revisions? Is there some more info or feedback you need?

tuomaseerola commented 1 year ago

@tuomaseerola can you update us on the status of the revisions? Is there some more info or feedback you need?

@faroit yes, the revisions are in 'R1' branch and with the exception of Sd corrected as SD (which I did now), the rest of issues were addressed earlier.

pmcharrison commented 1 year ago

Thanks @tuomaseerola, @faroit, I apologise for not keeping track of the conversation above about the location of the edits. @faroit, you ran generate PDF recently, but this will have generated the PDF for the wrong branch, no? It is hard for us to review the revisions without this. @tuomaseerola one way to facilitate the review would be if you create a PR comparing master with R1, then we can see the diff.

faroit commented 1 year ago

@tuomaseerola @pmcharrison i think the easiest would be to merge into main now for the pdf and track the changes in https://github.com/tuomaseerola/onsetsync/compare/master...R1

faroit commented 1 year ago

@mrocamora can you update us on the unchecked items from your review? have these been addressed so far?

mrocamora commented 1 year ago

Hi @faroit Yes, the community guidelines and the automated tests have been added. Should I close the two issues?

mrocamora commented 1 year ago

I am with @pmcharrison in that we cannot see the changes in the generated pdf. Once this is solved (@pmcharrison suggested a PR), I can check the minor typos and corrections.

faroit commented 1 year ago

@tuomaseerola can you please proceed with the merge so that we can create a new PDF for the reviewers?

tuomaseerola commented 1 year ago

@faroit sure, it was available as a PR for the last 3 weeks but now I did a merge to enable to the pdf compilation.

faroit commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

faroit commented 1 year ago

@pmcharrison @mrocamora the new pdf build is linked ☝️. Can you please update us, if your reviews have been addressed?

pmcharrison commented 1 year ago

Hi all, I'm very sorry that this has taken a while to sort out. I'm really happy with the revised paper and think it's good to go. I think it's going to be a really useful resource and inspiration for people working on this topic.

A minor note is that I think Figure 2 could be improved slightly:

image

The simplest improvement would be to reduce the % label size so that the labels don't overlap. That would make the plot look more production ready.

image

If I were to be more picky, I would say that the label representation is not the best way to represent the data. The point of making plots is that one should not have to resort to listing numbers like we do in tables. I think it would be preferable if these mean offsets were presented instead by a marginal line plot. I appreciate that this is a bit more complicated to achieve in R, but I believe you could achieve it after a bit of work via grid.arrange (see this example) https://stackoverflow.com/a/17649351.

All in all, though, I don't think this consideration is that important, and you should feel absolutely free to leave it as is.