[REVIEW]: Course Materials for an Introduction to Data-Driven Chemistry

whedon commented 1 year ago

Submitting author: @ppxasjsm (Antonia S J S Mey) Repository: https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry Version: 1.1.0 Editor: @arm61 Reviewer: @lucydot, @ghutchis Archive: 10.5281/zenodo.7782433

:warning: JOSE reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSE is currently operating in a "reduced service mode".

Status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/b4bb200d717653e896a84e2bbef8aa83"><img src="https://jose.theoj.org/papers/b4bb200d717653e896a84e2bbef8aa83/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/b4bb200d717653e896a84e2bbef8aa83/status.svg)](https://jose.theoj.org/papers/b4bb200d717653e896a84e2bbef8aa83)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@lucydot & @ghutchis, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/jose-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @arm61 know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @lucydot

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSE code of conduct.

General checks

[x] Repository: Is the source for this learning module available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
[x] Version: 1.1.0
[x] Authorship: Has the submitting author (@ppxasjsm) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

[x] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

[x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
[x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
[x] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
[x] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
[x] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
[x] Description: Does the paper describe the learning materials and sequence?
[x] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
[x] Could someone else teach with this module, given the right expertise?
[x] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @ghutchis

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSE code of conduct.

General checks

[x] Repository: Is the source for this learning module available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
[x] Version: 1.1.0
[x] Authorship: Has the submitting author (@ppxasjsm) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

[x] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

[x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
[x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
[x] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
[x] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
[x] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
[x] Description: Does the paper describe the learning materials and sequence?
[x] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
[x] Could someone else teach with this module, given the right expertise?
[x] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

whedon commented 1 year ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @lucydot, @ghutchis it looks like you're currently assigned to review this paper :tada:.

:warning: JOSE reduced service mode :warning:

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/jose-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/jose-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 1 year ago

Wordcount for paper.md is 1764

whedon commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon commented 1 year ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.33 s (160.5 files/s, 82405.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Jupyter Notebook                26              0          23251           1547
SVG                              5            128              2           1445
Markdown                        15             81              0            260
TeX                              1             14              0            221
YAML                             3             12             19             81
Python                           3             29             49             80
-------------------------------------------------------------------------------
SUM:                            53            264          23321           3634
-------------------------------------------------------------------------------

Statistical information for the repository 'f0731c738416b805f4758c4a' was
gathered on 2023/01/17.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
DEGIACOMI                        1            46              0            0.04
ppxasjsm                         6         59543          59431           99.96

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Matteo Degiacomi             46          100.0          0.0               23.91
ppxasjsm                    112            0.2          0.6               26.79

whedon commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.26434/chemrxiv.13656665.v1 is OK
- 10.1038/s41586-020-2649-2 is OK
- 10.1039/C8RP00105G is OK
- 10.1109/MCSE.2007.55 is OK
- 10.1021/bk-2021-1387.ch009 is OK
- 10.21105/jose.00032 is OK
- 10.1021/acs.jchemed.1c00142 is OK
- 10.1021/acs.jchemed.9b01131 is OK
- 10.21105/jose.00148 is OK
- 10.1021/bk-2020-1365.ch001 is OK
- 10.1021/acsinfocus.7e5030 is OK
- 10.5281/zenodo.7344967 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.21105/jose.00139 is OK

MISSING DOIs

- None

INVALID DOIs

- None

arm61 commented 1 year ago

Okay @lucydot and @ghutchis thanks for agreeing to review this interesting work! If you work through the checklist and if there is any problems/comments about the material, I would recommend opening an issue on the material repository and the authors can sort them out. More information about the review guidelines can be found on the Open Journals documentation pages: https://openjournals.readthedocs.io/en/jose/reviewer_guidelines.html

If anyone has any questions just ping me on here!

lucydot commented 1 year ago

Hello @ppxasjsm -

Great first impressions scanning through the resource.

I'm working through the checklist, will start today and will continue through till next week.

First issue (small points regarding documentation) raised here: https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/37

ppxasjsm commented 1 year ago

Thanks @lucydot! I will be on holidays next week, so I will be slow at responding. Just an FYI!

lucydot commented 1 year ago

@ppxasjsm - thanks for letting me know and no problem, will give me nice chunk of time to do initial look through 👁️

lucydot commented 1 year ago

Hi @ppxasjsm a quick heads up that I've raised two other issues on the repo. I'll add to the "Other small suggestions" as I work through the course.

arm61 commented 1 year ago

Well this interaction has made my week! Screenshot 2023-01-24 at 17 02 23

whedon commented 1 year ago

:wave: @lucydot, please update us on how your review is going (this is an automated reminder).

whedon commented 1 year ago

:wave: @ghutchis, please update us on how your review is going (this is an automated reminder).

ghutchis commented 1 year ago

Sorry - fell to the bottom of the pile. I should be able to finish the review / comments this week.

lucydot commented 1 year ago

@ppxasjsm I've updated https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/38 with a few other bits.

I really like the problem solving introduction to the course and will use this for my own teaching. The Mentimetre integration is also great.

Note that I found it hard to judge the suitability of some of the questions for a UG chemistry curriculum as I studied physics, and am only familiar with a very particular part of chemistry (translation: I don't understand most of UG chemistry).

Some thoughts for the future (not acceptance blockers for this review):

hosting this as an online book (using e.g. quarto) might improve readability. There is the option to integrate binder buttons so that it is not at the sake of user interaction.
would you consider including the slides that are referenced?
as a lecturer, one of the things I most struggle with is designing/writing summative assessment. It'd be great to understand a little more about how summative assessment fits in with this course.
I do see a bit of a disjoint between the parts of the course built from Software Carpentry resources (earlier chapters), and those that aren't (later chapters). The former have questions suitable for a more general audience, and explanations are split into smaller chunks (there are more headings). Exactly the same thing occured when I wrote a similar course for physicists, so I'm not really one to criticise. Ultimately, a course with domain specific examples throughout, and perfectly consistent writing style / lesson structure would be ideal. But I'd say thats a hypothetical end point to aim for rather than a requirement for a complete and effective course, which this appears to be. Clearly a lot of work has already gone into it as it is.

ppxasjsm commented 1 year ago

hosting this as an online book (using e.g. quarto) might improve readability. There is the option to integrate binder buttons so that it is not at the sake of user interaction.

Yes I've actually started looking into this, but then ran out of time to get an auto build for this for github actions...I can try and get this sorted for the review, but this will depend on when I have time to sort this out.

would you consider including the slides that are referenced?

Yes absolutely! I'll add them.

as a lecturer, one of the things I most struggle with is designing/writing summative assessment. It'd be great to understand a little more about how summative assessment fits in with this course.

Do you want this more generally or in the paper? We have two summative assessments after session 3 and 7 and then a mini workshop at the end that tests everything. I'd be happy to share one example of a summative assessment, but because we tend to reuse some of the material there are some restrictions around this.

I do see a bit of a disjoint between the parts of the course built from Software Carpentry resources (earlier chapters), and those that aren't (later chapters). The former have questions suitable for a more general audience, and explanations are split into smaller chunks (there are more headings). Exactly the same thing occured when I wrote a similar course for physicists, so I'm not really one to criticise. Ultimately, a course with domain-specific examples throughout, and perfectly consistent writing style / lesson structure would be ideal. But I'd say thats a hypothetical end point to aim for rather than a requirement for a complete and effective course, which this appears to be. Clearly a lot of work has already gone into it as it is.

Yes I totally agree with your point. I think the smaller headings kind of make sense early on and may be harder to keep up later on but this can be streamlined. For next year, I've started putting together some extra practice material that is geared towards the topics for the first 3 lessons for at home practice, but this isn't complete yet. I've also found it difficult to find 'truely' domain specific examples for teaching syntax etc...

lucydot commented 1 year ago

Hi @ppxasjsm

Yes I've actually started looking into this, but then ran out of time to get an auto build for this for github actions...I can try and get this sorted for the review, but this will depend on when I have time to sort this out.

I don't think it's required for the review, as the course can be followed clear enough as is.

Do you want this more generally or in the paper? We have two summative assessments after session 3 and 7 and then a mini workshop at the end that tests everything. I'd be happy to share one example of a summative assessment, but because we tend to reuse some of the material there are some restrictions around this.

If you are able to share an example that would be great, but again not necessary for the review. Though it'd be great to have a chat with you about assessment sometime! That's outside the scope of this review though.

ppxasjsm commented 1 year ago

Hi @lucydot and @ghutchis, just a heads up that if you have any comments on the content of the course I have some capacity to make edits in the next couple of weeks to address any issues you might want to raise! Thanks :)

arm61 commented 1 year ago

Hey @lucydot and @ghutchis, just checking in to see how things are going on the reviews. Looks like there is a lot of checkboxes filled, but just a couple still open. If you have particular comments/questions, ask them here or open an issue on the repository.

lucydot commented 1 year ago

Hi @ppxasjsm, I'll update the tick boxes following outcome on the discussions at https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/39 , https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/38 and https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/37 .

ghutchis commented 1 year ago

Thanks for the prompt - I'll come back and finish up my comments today.

ppxasjsm commented 1 year ago

I'll update the tick boxes following outcome on the discussions at https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/39 , https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/38 and https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/37 .

Great! Will try and fix these this week.

ppxasjsm commented 1 year ago

@whedon generate pdf

whedon commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ppxasjsm commented 1 year ago

@whedon generate pdf

whedon commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ppxasjsm commented 1 year ago

I decided to ditch binder. Updated paper to reflect only use of Colab.

@whedon generate pdf

ppxasjsm commented 1 year ago

@whedon generate pdf

whedon commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

ppxasjsm commented 1 year ago

https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/39

Fixed this. Now completely on colab. There is a bit of extra code for pulling data in, but should be useable.

ppxasjsm commented 1 year ago

@lucydot I think I have now addressed all comments in the issues you had opened. Please advise if you want me to make any further changes.

@ghutchis Please let me know if you have any other issues you would want to raise on the repository.

Thanks for all your useful comments so far everyone!

lucydot commented 1 year ago

@ppxasjsm I've just read through the three issues, and I agree all have been addressed. I'm happy to recommend this work for publication - and I plan to adapt some of this for use in my own teaching :star:

ppxasjsm commented 1 year ago

Thank you @lucydot. Glad you think the material may also be useful to you!

arm61 commented 1 year ago

That’s one review ticked off ✅. @ghutchis it looks like you are nearly done also?

ppxasjsm commented 1 year ago

Hi @ghutchis anything I can help with to finalise this review?

ghutchis commented 1 year ago

Yeah, sorry - slammed last week finishing a proposal and now getting caught up. I've set aside tomorrow afternoon (Pittsburgh time) to finish this off.

ppxasjsm commented 1 year ago

Amazing and thank you! I totally get the being very swamped with too many things problem!

ghutchis commented 1 year ago

Overall, I think you've fixed a lot of things I noted on my first assessment. I think my remaining issue would be: https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/50

ppxasjsm commented 1 year ago

@ghutchis with commit 4f8bad5 I have now addressed https://github.com/Edinburgh-Chemistry-Teaching/Data-driven-chemistry/issues/50. I think this means I have addressed all open queries remaining for this review.

ppxasjsm commented 1 year ago

Thank you @lucydot and @ghutchis for all your useful comments and suggestions to improve the material. It looks like @ghutchis is happy with the above commit! @arm61 Please let me know how to proceed now and thank you for such a pleasant review process here on Github!

arm61 commented 1 year ago

Nice work everyone. I will get started on the editorial stuff just now.

arm61 commented 1 year ago

@whedon generate pdf

whedon commented 1 year ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

arm61 commented 1 year ago

@whedon check references

whedon commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.26434/chemrxiv.13656665.v1 is OK
- 10.1038/s41586-020-2649-2 is OK
- 10.1039/C8RP00105G is OK
- 10.1109/MCSE.2007.55 is OK
- 10.1021/bk-2021-1387.ch009 is OK
- 10.21105/jose.00032 is OK
- 10.1021/acs.jchemed.1c00142 is OK
- 10.1021/acs.jchemed.9b01131 is OK
- 10.21105/jose.00148 is OK
- 10.1021/bk-2020-1365.ch001 is OK
- 10.1021/acsinfocus.7e5030 is OK
- 10.5281/zenodo.7344967 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.21105/jose.00139 is OK

MISSING DOIs

- 10.1109/mcse.2006.122 may be a valid DOI for title: Software Carpentry: Getting Scientists to Write Better Code by Making Them More Productive

INVALID DOIs

- None

arm61 commented 1 year ago

@ppxasjsm two things in the references for resolve:

The author for pandas renders pretty weird at the moment (I think cause of the curlys) can you have a look.
The DOI is missing for the software carpentry stuff. The DOI that the bot (who we shall not name) has found seems like the right one.

I will get started on reading the paper now for typos and the like.

ppxasjsm commented 1 year ago

Thank you @arm61! I am working on the DOI/references stuff.

ppxasjsm commented 1 year ago

@whedon check references

ppxasjsm commented 1 year ago

@whedon generate pdf

whedon commented 1 year ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.26434/chemrxiv.13656665.v1 is OK
- 10.1038/s41586-020-2649-2 is OK
- 10.1039/C8RP00105G is OK
- 10.1109/MCSE.2007.55 is OK
- 10.1021/bk-2021-1387.ch009 is OK
- 10.21105/jose.00032 is OK
- 10.1021/acs.jchemed.1c00142 is OK
- 10.1021/acs.jchemed.9b01131 is OK
- 10.21105/jose.00148 is OK
- 10.1021/bk-2020-1365.ch001 is OK
- 10.1021/acsinfocus.7e5030 is OK
- 10.5281/zenodo.7344967 is OK
- 10.1038/s41592-019-0686-2 is OK
- 10.21105/jose.00139 is OK
- 10.1109/mcse.2006.122 is OK

MISSING DOIs

- None

INVALID DOIs

- None

openjournals / jose-reviews