[REVIEW]: Powering single-cell analyses in the browser with WebAssembly

editorialbot commented 1 year ago

Submitting author: !--author-handle-->@LTLA@AoifeHughes<!--end-editor-- Reviewers: @yongrenjie, @llewelld Archive: 10.5281/zenodo.8273645

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/9c92da27671e55617137f1f314bc48be"><img src="https://joss.theoj.org/papers/9c92da27671e55617137f1f314bc48be/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/9c92da27671e55617137f1f314bc48be/status.svg)](https://joss.theoj.org/papers/9c92da27671e55617137f1f314bc48be)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@yongrenjie & @llewelld, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @AoifeHughes know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @yongrenjie

editorialbot commented 1 year ago

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot commented 1 year ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.11 s (809.7 files/s, 229450.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JavaScript                      59           1637            982          19601
CSS                             16            177             58           1006
TeX                              2             69              0            623
Markdown                         5             74              0            222
JSON                             2              0              0            119
YAML                             1              5              3             30
HTML                             1              7             21             19
SVG                              1              0              0              1
-------------------------------------------------------------------------------
SUM:                            87           1969           1064          21621
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

editorialbot commented 1 year ago

Wordcount for paper.md is 1504

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/s41592-019-0654-x is OK
- 10.1038/s41592-019-0654-x is OK
- 10.1137/04060593X is OK
- 10.2307/2346830 is OK
- 10.1109/TIT.1982.1056489 is OK
- 10.1093/nar/gkv007 is OK
- 10.1080/01621459.1979.10481038 is OK
- 10.21105/joss.00861 is OK
- 10.12688/f1000research.9501.2 is OK
- 10.1093/bioinformatics/btw777 is OK
- 10.1371/journal.pcbi.1006135 is OK
- 10.1145/3140587.3062363 is OK
- 10.1038/nmeth.3615 is OK
- 10.1101/118992 is OK
- 10.1093/bioinformatics/btt100 is OK
- 10.1101/2021.04.05.438318 is OK
- 10.1038/nmeth.3038 is OK
- 10.1038/nrg3833 is OK
- 10.1038/s41467-017-02001-5 is OK
- 10.1038/s41467-019-09182-1 is OK
- 10.1016/j.immuni.2019.03.009 is OK
- 10.1038/nbt.4091 is OK
- 10.1038/s41590-018-0276-y is OK
- 10.1186/s13059-018-1486-1 is OK
- 10.3233/ida-2007-11402 is OK
- 10.1186/s13059-016-0947-7 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

AoifeHughes commented 1 year ago

Hi all, thanks for being here ant thank you for being part of this process. Please feel free to ping me here if there's any problems, as the bot said in its initial comment, it would be great if reviewers could

@editorialbot generate my checklist

Please use this issue as a place to speak openly with the author(s) and make comments, if you open issues on the original repo, please link them here so it's easier for me to check up on them.

Any other issues, or questions please just drop me a message here 😄

yongrenjie commented 1 year ago

Review checklist for @yongrenjie and @llewelld

Conflict of interest

[x] We confirm that we have read the JOSS conflict of interest (COI) policy and that: we have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] We confirm that we read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the https://github.com/kanaverse/kana?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
- See issue (f) below.
[x] Contribution and authorship: Has the submitting author (@LTLA) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
[x] Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
[x] Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
[x] Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
- See issue (i) below.
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)
- See issue (b) below.

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
- See issue (g) below.
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
- See issue (h) below.

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
- See issue (c) below.
[x] A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
- See issue (d) below.
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
- See issue (e) below.
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

AoifeHughes commented 1 year ago

Just bumping this, I've had chats with the reviewers and I know they're working on it. :)

llewelld commented 1 year ago

Thanks for the bump @AoifeHughes. Just to confirm, @yongrenjie and I have been working on this and our review is in the pipeline.

yongrenjie commented 1 year ago

Review — by @llewelld and @yongrenjie

There are numerous contributions from the two authors of the paper (Aaron T. L. Lun and Jayaram Kancherla) in the form of git commits going back to 2021. They are clearly the main authors of kana and scran.js. They are also significant contributors to scran-cli and scran, although in the case of scran not the only contributors.

The scholarly effort appears significant and certainly satisfies the JOSS guidelines. Although it's hard for us to place this within the context of the field in terms of new research challenges, given our background, it certainly appears that way.

The functionality of the web app is as described in the paper, and closely follows that of the 'Basics of Single-Cell Analysis with Bioconductor' book written by Lun and others. As we do not work in this area, we do not have our own datasets, but we have tested the software with some of the example datasets provided by the authors. Although some bugs still remain in the web app (for example, we ran into an error when selecting clusters with numerical values; and the clusters disappear when switching tabs in the browser), it appears from the repository issues that the authors are already aware of these, and it does not in any way detract from the core functionality of the software.

The contribution of the work does not depend on data access. As mentioned above, there are example files provided as GitHub Releases, which certainly contribute to the ease of access of the software, but these are not themselves contributions provided by the authors. The example files are easily accessible and installed as part of the software.

The software does not appear to be providing, in and of itself, original results. There is other software that performs similar tasks (some written by the authors themselves), but lacking the client-side browser-based approach.

The main contribution is therefore the transfer of the software from a command-line or R-based toolset to offering browser-based access, easy installation and removing the need for server-side computation.

The example datafiles provided do include data research on animals, but this data is not new and is a standard dataset that is widely available. Therefore, issues surrounding the animal research policy would not seem to be relevant here.

The writeup is excellent: well-written, concise and clear. We found the screenshots to also be of high quality: nicely produced and of great help in giving an immediate impression and understanding of the tool.

The C++ code is well-written, cleanly structured and styled, makes good use of the standard libraries and is largely self-contained. We had expected to find more external dependencies not written by the authors themselves, but this in itself is not an issue. We focused on this part of the code as it is where the analysis mainly happens; it's clearly a substantial piece of work.

Overall, this is a solid piece of open source scientific software that definitely feels worthy of inclusion in JOSS. The write-up is clear, but missing just a few pieces to round it off. Having addressed these, we feel confident that the submission should ultimately be accepted.

We have created separate issues in the authors' repositories for each of these points:

The paper: We created a few issues which each correspond to to remarks on the text itself. Please see issues (a) through (e).
License: looking at the licence of the app threw up one query, please see issue (f).
Unit tests: The kana user-interface app doesn't have unit tests, or test steps. However the scran.js library, which it uses, does. We weren't able to get the unit tests working for scran.js, seee issue (g).
Community guidelines: see issue (h).
Deployment: we also hit some problems. Please see issue (i).
Documentation: please see issue (j).

Links to issues:

Category	Label in review	Link
Paper	(a)	~~https://github.com/kanaverse/kana/issues/231~~
	(b)	https://github.com/kanaverse/kana/issues/232
	(c)	~~https://github.com/kanaverse/kana/issues/233~~
	(d)	~~https://github.com/kanaverse/kana/issues/234~~
	(e)	~~https://github.com/kanaverse/kana/issues/235~~
License	(f)	~~https://github.com/LTLA/libscran/issues/59~~
Unit tests	(g)	~~https://github.com/kanaverse/scran.js/issues/80~~
Community guidelines	(h)	~~https://github.com/kanaverse/kana/issues/236~~
Deployment	(i)	~~https://github.com/kanaverse/kana/issues/237 and related PR, https://github.com/kanaverse/kana/pull/230~~
Documentation	(j)	~~https://github.com/kanaverse/kana/issues/238~~

LTLA commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

LTLA commented 1 year ago

Thanks for the kind words and helpful comments @llewelld @yongrenjie.

I think we've addressed most of the comments, aside from a few that need some clarification (whether we can run keep rolling past the word limit in kanaverse/kana#232, some missing review text in kanaverse/kana#234).

I haven't closed any of the issues yet; I'll leave that to the reviewers if they're satisfied with our responses.

LTLA commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

AoifeHughes commented 1 year ago

Morning 😊, I see a lot of the reviewer's issues are still open and ongoing. Is everything progressing okay? Please do let me know if I cab be of any help!

LTLA commented 1 year ago

@AoifeHughes I think we're making good progress. Maybe you could give your thoughts in kanaverse/kana#232?

LTLA commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

AoifeHughes commented 1 year ago

Okie dokie, I’ll need to double check a few things. Thanks for linking to the issue!

LTLA commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

jkanche commented 1 year ago

just a friendly bump to see if there are any more comments or issues.

llewelld commented 1 year ago

A quick recap on where things stand as I see it. All of the issues now look good to me. I think we are still waiting on a final decision in relation to #232. @yongrenjie and I will have to go through the checklist above again; it looks like everything should be resolved there already, but we should still complete the process as a formality. There may be a slight delay before I can do this (my apologies) since we hit August with vacations.

@yongrenjie: would you be available to take a look at this together next week?

yongrenjie commented 1 year ago

It seems like it should be all good — the only outstanding issue is https://github.com/kanaverse/kana/issues/232, where we're just waiting for confirmation re. citing the preprint (though my opinion is that it shouldn't be a problem and we can probably close it soon).

I've ticked off the items in the checklist for now.

@AoifeHughes Is there anything else we need to do as reviewers, apart from recommending that the paper be accepted in its current form 😉?

AoifeHughes commented 1 year ago

@yongrenjie, @llewelld thanks both for your work on this! Nothing extra needed from you two, but let me know when you feel it's ready for a final check from me. Once I go over everything I can do the final checks for organising DOIs and proceeding with publishing (after EiC has a check too)

llewelld commented 1 year ago

Thanks @AoifeHughes for all of your work and help with this too. The review checklist is complete and @yongrenjie and I are happy with the changes. It was great to get the opportunity to read and review it. Thanks @LTLA and @jkanche for your excellent work and responsiveness throughout the review process.

AoifeHughes commented 1 year ago

@LTLA At this point could you:

[x] Make a tagged release of your software, and list the version tag of the archived version here.
[x] Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
[x] Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
[x] Please list the DOI of the archived version here.

I can then move forward with recommending acceptance of the submission.

jkanche commented 1 year ago

@AoifeHughes We already have a tagged v3.0.0 release of Kana and made the submission on zenodo: https://zenodo.org/record/8273506

Let me know if this works for you.

AoifeHughes commented 1 year ago

Hi @jkanche, the version 3.0.0 release doesn't contain any changes which have been made since April 17th. Could you update to a release which contains these and also update the Zenodo record accordingly

jkanche commented 1 year ago

Tagged a new release (v3.0.1) and the zenodo link should reflect these changes.

AoifeHughes commented 1 year ago

@editorialbot set 10.5281/zenodo.8273645 as archive

editorialbot commented 1 year ago

Done! archive is now 10.5281/zenodo.8273645

AoifeHughes commented 1 year ago

@editorialbot set v3.0.1 as version

editorialbot commented 1 year ago

Done! version is now v3.0.1

AoifeHughes commented 1 year ago

@editorialbot recommend-accept

editorialbot commented 1 year ago

Attempting dry run of processing paper acceptance...

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/s41592-019-0654-x is OK
- 10.1038/s41592-019-0654-x is OK
- 10.1137/04060593X is OK
- 10.2307/2346830 is OK
- 10.1109/TIT.1982.1056489 is OK
- 10.1093/nar/gkv007 is OK
- 10.1080/01621459.1979.10481038 is OK
- 10.21105/joss.00861 is OK
- 10.12688/f1000research.9501.2 is OK
- 10.1093/bioinformatics/btw777 is OK
- 10.1371/journal.pcbi.1006135 is OK
- 10.1145/3140587.3062363 is OK
- 10.1038/nmeth.3615 is OK
- 10.1101/118992 is OK
- 10.1093/bioinformatics/btt100 is OK
- 10.1101/2021.04.05.438318 is OK
- 10.1038/nmeth.3038 is OK
- 10.1038/nrg3833 is OK
- 10.1038/s41467-017-02001-5 is OK
- 10.1038/s41467-019-09182-1 is OK
- 10.1016/j.immuni.2019.03.009 is OK
- 10.1038/nbt.4091 is OK
- 10.1038/s41590-018-0276-y is OK
- 10.1186/s13059-018-1486-1 is OK
- 10.3233/ida-2007-11402 is OK
- 10.1186/s13059-016-0947-7 is OK
- 10.1101/2022.03.02.482701 is OK

MISSING DOIs

- None

INVALID DOIs

- None

editorialbot commented 1 year ago

:wave: @openjournals/bcm-eics, this paper is ready to be accepted and published.

Check final proof :point_right::page_facing_up: Download article

If the paper PDF and the deposit XML files look good in https://github.com/openjournals/joss-papers/pull/4500, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

jkanche commented 1 year ago

Thanks @AoifeHughes for your help in this process, Is there anything else we need to do?

Kevin-Mattheus-Moerman commented 1 year ago

@LTLA I am the AEiC for this track and here to help with final steps. I have check this review, your repository, the archive, and the paper. Most seems in order. I only have the below points that require your attention:

[x] Please edit your archive listed software license to match your actual software license. It currently says Creative Commons Attribution 4.0 International but it should refer to your MIT license.
[x] In your affiliations please spell out USA as United States of America.

It looks like these DOIs are missing:

LTLA commented 1 year ago

Thanks @Kevin-Mattheus-Moerman, we've implemented the requested changes.

LTLA commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Kevin-Mattheus-Moerman commented 1 year ago

@editorialbot accept

editorialbot commented 1 year ago

Doing it live! Attempting automated processing of paper acceptance...

editorialbot commented 1 year ago

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

``` cff-version: "1.2.0" authors: - family-names: Lun given-names: Aaron T. L. orcid: "https://orcid.org/0000-0002-3564-4813" - family-names: Kancherla given-names: Jayaram orcid: "https://orcid.org/0000-0001-5855-5031" doi: 10.5281/zenodo.8273645 message: If you use this software, please cite our article in the Journal of Open Source Software. preferred-citation: authors: - family-names: Lun given-names: Aaron T. L. orcid: "https://orcid.org/0000-0002-3564-4813" - family-names: Kancherla given-names: Jayaram orcid: "https://orcid.org/0000-0001-5855-5031" date-published: 2023-09-05 doi: 10.21105/joss.05603 issn: 2475-9066 issue: 89 journal: Journal of Open Source Software publisher: name: Open Journals start: 5603 title: Powering single-cell analyses in the browser with WebAssembly type: article url: "https://joss.theoj.org/papers/10.21105/joss.05603" volume: 8 title: Powering single-cell analyses in the browser with WebAssembly ```

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

editorialbot commented 1 year ago

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

editorialbot commented 1 year ago

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

Check final PDF and Crossref metadata that was deposited :point_right: https://github.com/openjournals/joss-papers/pull/4528
Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.05603
If everything looks good, then close this review issue.
Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

Kevin-Mattheus-Moerman commented 1 year ago

@LTLA congratulations on this publication!

Thanks for editing @AoifeHughes !

And a special thanks to the reviewers: @yongrenjie, @llewelld

openjournals / joss-reviews