[REVIEW]: A Data Carpentry- Style Metagenomics Workshop

whedon commented 1 year ago

Submitting author: !--author-handle-->@nselem@JasonJWilliamsNY@enkera<!--end-reviewers-list-- Archive: 10.5281/zenodo.8306876 Paper kind: learning module

Status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/0b49e5dbfe3fadb4743408dc3d6fb99f"><img src="https://jose.theoj.org/papers/0b49e5dbfe3fadb4743408dc3d6fb99f/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/0b49e5dbfe3fadb4743408dc3d6fb99f/status.svg)](https://jose.theoj.org/papers/0b49e5dbfe3fadb4743408dc3d6fb99f)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@enkera, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/jose-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @JasonJWilliamsNY know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @enkera

Conflict of interest

[ ] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[ ] I confirm that I read and will adhere to the JOSE code of conduct.

General checks

[ ] Repository: Is the source for this learning module available at the repository url?
[ ] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
[ ] Version: Does the release version given match the repository release (v1.0.0)?
[ ] Authorship: Has the submitting author (@nselem) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

[ ] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
[ ] Installation instructions: Is there a clearly stated list of dependencies?
[ ] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
[ ] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

[ ] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
[ ] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
[ ] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
[ ] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
[ ] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

[ ] Authors: Does the paper.md file include a list of authors with their affiliations?
[ ] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
[ ] Description: Does the paper describe the learning materials and sequence?
[ ] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
[ ] Could someone else teach with this module, given the right expertise?
[ ] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

whedon commented 1 year ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @enkera it looks like you're currently assigned to review this paper :tada:.

:warning: JOSE reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSE is currently operating in a "reduced service mode".

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/jose-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/jose-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 1 year ago

Wordcount for paper.md is 1504

whedon commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon commented 1 year ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.09 s (498.6 files/s, 48211.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Markdown                        26            372              0           1427
Python                           7            342            230           1063
TeX                              1             22              0            233
R                                4             52             39            161
YAML                             3             30             54            126
make                             1             28             29             67
JSON                             1              0              0             57
Ruby                             1              3              1              7
Bourne Shell                     1              2              2              4
-------------------------------------------------------------------------------
SUM:                            45            851            355           3145
-------------------------------------------------------------------------------

Statistical information for the repository '1ac5fffa9be4e8bd17f6359c' was
gathered on 2023/06/01.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Andy Boughton                    1             5              4            0.03
Francois Michonneau              1          2306              0            6.45
François Michonneau              5            22            130            0.43
Gabriel A. Devenyi               1             5              3            0.02
Greg Wilson                     47         12802          13284           73.01
Joseph Stachelek                 1             1              1            0.01
Katrin Leinweber                 3             6              6            0.03
Maxim Belkin                    14            55             55            0.31
Michael R. Crusoe                1            74             70            0.40
Raniere Silva                   20           262            687            2.66
Rémi Emonet                      2            49             17            0.18
Tracy Teal                       4          3817           1950           16.14
W. Trevor King                   1             0              1            0.00
William L. Close                 3            53             56            0.31
naught101                        1             4              2            0.02

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Andy Boughton                 3           60.0         10.3                0.00
François Michonneau          95          431.8          9.8                0.00
Greg Wilson                 506            4.0         32.1               11.26
Joseph Stachelek              1          100.0         10.2                0.00
Katrin Leinweber              1           16.7         10.0                0.00
Maxim Belkin                 46           83.6         10.3                2.17
Michael R. Crusoe            62           83.8         11.2                4.84
Raniere Silva                58           22.1         13.4               10.34
Tracy Teal                  873           22.9         20.1               12.37
naught101                     1           25.0         16.3                0.00

whedon commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1051/0004-6361/201629272 is OK
- 10.1051/0004-6361/201322068 is OK
- 10.5281/ZENODO.3585993 is OK
- 10.1101/2021.06.17.448726 is OK
- 10.1101/2021.06.15.448091 is OK
- 10.21105/jose.00049 is OK
- 10.7554/eLife.49816 is OK
- 10.5281/ZENODO.3260309 is OK
- 10.3389/fmicb.2020.578600 is OK

MISSING DOIs

- 10.21105/jose.00049 may be a valid DOI for title: A graduate student-led participatory live-coding quantitative methods course in R: Experiences on initiating, developing, and teaching
- 10.21105/jose.00027 may be a valid DOI for title: An Introduction to Applied Bioinformatics: a free, open, and interactive text.
- 10.21105/jose.00021 may be a valid DOI for title: CFD Python: the 12 steps to Navier-Stokes equations
- 10.3389/fmicb.2020.578600 may be a valid DOI for title: A Curricular Bioinformatics Approach to Teaching Undergraduates to Analyze Metagenomic Datasets Using R
- 10.1002/bmb.21387 may be a valid DOI for title: Incubators: Building community networks and developing open educational resources to integrate bioinformatics into life science education
- 10.1525/abt.2021.83.1.33 may be a valid DOI for title: Introducing Python Programming into Undergraduate Biology

INVALID DOIs

- None

whedon commented 1 year ago

:wave: @enkera, please update us on how your review is going (this is an automated reminder).

nselem commented 1 year ago

Hello @JasonJWilliamsNY and @enkera Please could you share if there is any news in the review process?

enkera commented 1 year ago

Hello @JasonJWilliamsNY. I've had a look at the workshop and it satisfies most of the criteria needed for publication. There seem to be several broken or indirect URL links, in particular the Instructor Notes, Setup, and Metagenomics Workshop Setup page links are dead or not working properly. Other than that, everything looks good.

nselem commented 1 year ago

Hello @enkera , We have updated Setup and instructor notes in Metagenomics Workshop, I think they broke when we change the repo from carpentries-incubator to carpentries-lab, thank you for noticing that.

nselem commented 1 year ago

Hello, @enkera and @JasonJWilliamsNY Is it something else needed for the workshop to be accepted?

JasonJWilliamsNY commented 1 year ago

Hi @nselem I'll take one last look so we can move to the acceptance step. Thanks @enkera for review!

JasonJWilliamsNY commented 1 year ago

@enkera can you check off the items in the checklist at the top of this page; don't worry if some don't apply.

JasonJWilliamsNY commented 1 year ago

@whedon unassign @enkera as reviewer

JasonJWilliamsNY commented 1 year ago

@editorialbot remove @enkera from reviewers

editorialbot commented 1 year ago

@enkera added to the reviewers list!

JasonJWilliamsNY commented 1 year ago

@enkera I removed and added you - if that does not work, I will consult with @labarba

enkera commented 1 year ago

I still can't check the boxes unfortunately.

On Wed, Jul 26, 2023 at 11:00 AM Jason Williams @.***> wrote:

@enkera https://github.com/enkera I removed and added you - if that does not work, I will consult with @labarba https://github.com/labarba

— Reply to this email directly, view it on GitHub https://github.com/openjournals/jose-reviews/issues/209#issuecomment-1651988209, or unsubscribe https://github.com/notifications/unsubscribe-auth/AETX4ADNFYTE4C4CCAZXHA3XSEWJTANCNFSM6AAAAAAYXBTHXI . You are receiving this because you were mentioned.Message ID: @.***>

JasonJWilliamsNY commented 1 year ago

@labarba can you help us out here?

arfon commented 1 year ago

@JasonJWilliamsNY – I would suggest that @enkera generates a new checklist with @editorialbot by typing the following in a new comment: @editorialbot generate my checklist

JasonJWilliamsNY commented 1 year ago

Pinging @enkera and will follow up by email.

enkera commented 1 year ago

@editorialbot generate my checklist

labarba commented 1 year ago

@openjournals/dev – On this review thread, reviewer @enkera was added before we moved JOSE to the new editorial workflow and bot, and I suspect he didn't accept the invitation in time to get the necessary permissions to edit the review auto-generated checklist. He now tried the command @editorialbot generate my checklist to get a new one, but it didn't work. Can you help us?

arfon commented 1 year ago

I can't see anything obviously wrong here @xuanxu. Any thoughts?

nniiicc commented 1 year ago

Just a heads up - we're having the same issue in #221

xuanxu commented 1 year ago

There was a problem with the bot not knowing which checklist to create (software or learning module review). It should be fixed now.

@enkera: can you try again the @editorialbot generate my checklist command?

enkera commented 1 year ago

Review checklist for @enkera

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSE code of conduct.

General checks

[x] Repository: Is the source for this learning module available at the https://github.com/carpentries-lab/metagenomics-workshop?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
[x] Version: Does the release version given match the repository release?
[x] Authorship: Has the submitting author (@nselem) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

[x] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

[x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
[x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
[x] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
[x] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
[x] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
[x] Description: Does the paper describe the learning materials and sequence?
[x] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
[x] Could someone else teach with this module, given the right expertise?
[x] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

JasonJWilliamsNY commented 1 year ago

@editorialbot generate pdf

editorialbot commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

JasonJWilliamsNY commented 1 year ago

@editorialbot check references

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1051/0004-6361/201629272 is OK
- 10.1051/0004-6361/201322068 is OK
- 10.5281/ZENODO.3585993 is OK
- 10.1101/2021.06.17.448726 is OK
- 10.1101/2021.06.15.448091 is OK
- 10.21105/jose.00049 is OK
- 10.7554/eLife.49816 is OK
- 10.5281/ZENODO.3260309 is OK
- 10.3389/fmicb.2020.578600 is OK

MISSING DOIs

- 10.21105/jose.00049 may be a valid DOI for title: A graduate student-led participatory live-coding quantitative methods course in R: Experiences on initiating, developing, and teaching
- 10.21105/jose.00027 may be a valid DOI for title: An Introduction to Applied Bioinformatics: a free, open, and interactive text.
- 10.21105/jose.00021 may be a valid DOI for title: CFD Python: the 12 steps to Navier-Stokes equations
- 10.3389/fmicb.2020.578600 may be a valid DOI for title: A Curricular Bioinformatics Approach to Teaching Undergraduates to Analyze Metagenomic Datasets Using R
- 10.1002/bmb.21387 may be a valid DOI for title: Incubators: Building community networks and developing open educational resources to integrate bioinformatics into life science education
- 10.1525/abt.2021.83.1.33 may be a valid DOI for title: Introducing Python Programming into Undergraduate Biology

INVALID DOIs

- None

JasonJWilliamsNY commented 1 year ago

Hi @ @nselem I have made a few optional suggestions via a pull request https://github.com/carpentries-lab/metagenomics-workshop/pull/20

Once you are ready, the next step is for you to create an archive on either https://zenodo.org/ or https://figshare.com/ and obtain a DOI (which you'll need to give to us). I can then move to the final acceptance and publication steps.

Tip: Here is a nice guide on doing the above with Zenodo: https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content#:~:text=Making%20Your%20Code%20Citable%201%20Choose%20your%20repository.,a%20shiny%20new%20badge%20showing%20your%20new%20DOI%21

labarba commented 1 year ago

Do note that the instructions provided in the link above result in the following behavior:

Zenodo archives your repository and issues a new DOI each time you create a new GitHub release.

This may or may not be behavior that you want for your project. Also note that the automatic Zenodo archives are titled with your repository name, and list every committer to the repo as an author. We request that authors edit the metadata of the Zenodo deposit so title and author list match the JOSE paper. It's just cleaner that way as readers see these as part of the "same scholarly object."

Czirion commented 1 year ago

Thank you @JasonJWilliamsNY and @labarba

We had already made these archives in Zenodo for each repository of the lesson: Metagenomics R: https://zenodo.org/badge/latestdoi/315204217 Metagenomics Analysis: https://zenodo.org/badge/latestdoi/287395428 Metagenomics shell: https://zenodo.org/badge/latestdoi/344547242 Metagenomics organization: https://zenodo.org/badge/latestdoi/349858841 Metagenomics workshop: https://zenodo.org/badge/latestdoi/315250269

Are those what you need? Or should I make something differently? They do have long lists of authors, I'll see if it can be edited without affecting the DOI.

JasonJWilliamsNY commented 1 year ago

I think you can issue a new version; we try and make sure the Zenodo and the paper authors are the same. Since this is new maybe @labarba can comment.

nselem commented 1 year ago

Yes I think you are right Jason, because our links conserve as authors "Raniere Silva; François Michonneau; Maxim Belkin; Nelly Sélem; Andy Boughton; Claudia Zirión Martínez; Erin Becker; Abby Cabunoc Mayes; Katrin Leinweber @gitlabhq; W. Trevor King; Aaron O'Leary; maneesha sane; Toby Hodges; João Rodrigues; Diego Garfias Gallegos; Rayna M Harris; Jonah Duckles; Kate Hertweck; John Blischak; Trevor Bekolay; Jemma Stachelek; AbrahamAvelar; Piotr Banaszkiewicz; aaronejaime; Edward Wallace; David Mawdsley; Ivan Gonzalez; Naupaka Zimmerman; Zhian N. Kamvar; Michael R. Crusoe" and we need to keep only the authors of this lesson.

"

labarba commented 1 year ago

To change the list of authors, you edit the metadata of the archive, which does not issue a new DOI.

Czirion commented 1 year ago

Thank you all, I have edited the list of authors in all the Zenodo archives. The links are the ones above.

labarba commented 1 year ago

"all the Zenodo archives"—I'm confused. We can only link the paper to one Zenodo archive. Are the materials spread out over several archives for this work?

Czirion commented 1 year ago

The contents of the lesson itself are spread out in several GitHub repositories and each of those has a Zenodo archive. The paper.md is in the landing repo of the lesson, whose Zenodo archive is https://zenodo.org/record/7713380

labarba commented 1 year ago

Is it possible to make one fat .zip with everything and deposit in its own Zenodo archive?

Czirion commented 1 year ago

I did this Zenodo archive https://doi.org/10.5281/zenodo.8306876

Does it work?

labarba commented 1 year ago

@editorialbot set 10.5281/zenodo.8306876 as archive

editorialbot commented 1 year ago

Done! archive is now 10.5281/zenodo.8306876

labarba commented 1 year ago

@JasonJWilliamsNY – when this is ready to be published, please run the recommend-accept command

JasonJWilliamsNY commented 1 year ago

@editorialbot recommend-accept

editorialbot commented 1 year ago

Attempting dry run of processing paper acceptance...

editorialbot commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1051/0004-6361/201629272 is OK
- 10.1051/0004-6361/201322068 is OK
- 10.5281/ZENODO.3585993 is OK
- 10.1101/2021.06.17.448726 is OK
- 10.1101/2021.06.15.448091 is OK
- 10.21105/jose.00049 is OK
- 10.7554/eLife.49816 is OK
- 10.5281/ZENODO.3260309 is OK
- 10.3389/fmicb.2020.578600 is OK

MISSING DOIs

- 10.21105/jose.00049 may be a valid DOI for title: A graduate student-led participatory live-coding quantitative methods course in R: Experiences on initiating, developing, and teaching
- 10.21105/jose.00027 may be a valid DOI for title: An Introduction to Applied Bioinformatics: a free, open, and interactive text.
- 10.21105/jose.00021 may be a valid DOI for title: CFD Python: the 12 steps to Navier-Stokes equations
- 10.3389/fmicb.2020.578600 may be a valid DOI for title: A Curricular Bioinformatics Approach to Teaching Undergraduates to Analyze Metagenomic Datasets Using R
- 10.1002/bmb.21387 may be a valid DOI for title: Incubators: Building community networks and developing open educational resources to integrate bioinformatics into life science education
- 10.1525/abt.2021.83.1.33 may be a valid DOI for title: Introducing Python Programming into Undergraduate Biology

INVALID DOIs

- None

editorialbot commented 1 year ago

The paper's PDF and metadata files generation produced some warnings that could prevent the final paper from being published. Please fix them before the end of the review process.

citation becker_datacarpentrygenomics-workshop:2019 not found

editorialbot commented 1 year ago

:warning: Error preparing paper acceptance. The generated XML metadata file is invalid.

IDREFS attribute rid references an unknown ID "ref-becker_datacarpentrygenomics-workshopU003A2019"

JasonJWilliamsNY commented 1 year ago

:warning: Error preparing paper acceptance. The generated XML metadata file is invalid.
IDREFS attribute rid references an unknown ID "ref-becker_datacarpentrygenomics-workshopU003A2019"

@labarba any clues? I'm not at a computer now but can look into later if not resolved.

openjournals / jose-reviews