[REVIEW]: PupillometryR: An R package for preparing and analysing pupillometry data

whedon commented 4 years ago

Submitting author: @samhforbes (Samuel Forbes) Repository: https://github.com/samhforbes/PupillometryR Version: v0.0.3 Editor: @oliviaguest Reviewer: @paulinepalma, @Athanasiamo Archive: 10.5281/zenodo.3897291

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/65fdd6bc59eb558543282e93140eb31e"><img src="https://joss.theoj.org/papers/65fdd6bc59eb558543282e93140eb31e/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/65fdd6bc59eb558543282e93140eb31e/status.svg)](https://joss.theoj.org/papers/65fdd6bc59eb558543282e93140eb31e)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@paulinepalma & @Athanasiamo, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @oliviaguest know.

✨ Please try and complete your review in the next six weeks ✨

Review checklist for @paulinepalma

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@samhforbes) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @Athanasiamo

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@samhforbes) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @paulinepalma, @Athanasiamo it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.12688/wellcomeopenres.15191.1 is OK
- 10.3758/s13428-017-1007-2 is OK
- 10.3758/s13428-018-01190-1 is OK
- 10.31234/osf.io/gvcxb is OK
- 10.1007/978-1-4757-7107-7 is OK
- 10.1111/j.1467-7687.2008.00805.x is OK
- 10.3758/s13428-011-0109-5 is OK
- 10.1177/2331216519832483 is OK
- 10.3758/s13423-018-1432-y is OK
- 10.1177/2331216518800869 is OK
- 10.1126/science.143.3611.1190 is OK
- 10.1037/h0026952 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

oliviaguest commented 4 years ago

👋 @paulinepalma, @Athanasiamo: Any code-related questions feel free to open issues on the package's repo itself and then link to whatever issue you open from here. Everything important about the PDF or the package, like high-level questions to me or @samhforbes, discussions and feedback, just leave it here as a comment directly. ✨ 🌷

drmowinckels commented 4 years ago

General checks

❌ License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?

LICENCE file does not contain any information on the type of licence to apply to the package.
recommendation: utilise usethis::use_[xx]_licence to create a valid licence file.
Noted in this isse ✅ Contribution and authorship: Has the submitting author (@samhforbes) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
One other contributor in a single minor commit. This looks ok to me.
Another package author is listed without any commit history. This person is mentioned in the paper acknowledgement.

Documentation

✅ Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.

since the package is also on CRAN, installation instructions should clearly state that, and only point to github installation for development versions. I had to read the paper to know it was on CRAN. noted in this issue ➖ Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
yes but not in an ideal way. in my opinion.
Great with data shipped with the package so one can give the functions a go.
in my opinion, the readme as a whole should be moved into a vignette folder and be compiled as such. This will also then run the vignette code during build checks, making sure that the vignette is actually executable (easiest way to do so it to use usethis::use_vignette() ). Noted in this issue
please add link to Sylvains tutorial in the README Noted in this issue
A new readme just highlighting some core functionality of the package can be used in stead.
Instead of informing about patch updates ( like the top of the current readme), use a NEWS file instead for this sort of information. noted in this issue ✅ Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)? Good
✅ Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
function examples are run on build checks, and everything checks out fine.
But there are no CI checks.
❌ Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support.
There is no such information, as noted in this issue

oliviaguest commented 4 years ago

@Athanasiamo thank you for this, can you also check/tick them off above in the OP as well?

oliviaguest commented 4 years ago

@openjournals/joss-editors @openjournals/joss-eics has anybody seen a case where the reviewer cannot tick/check the boxes above? @Athanasiamo is not able to interact with the OP, is there a simple solution we are not aware of? She is logged in and I have sent her a video of what it looks like from my end. I have no idea what is wrong — thanks.

danielskatz commented 4 years ago

@whedon re-invite @Athanasiamo as reviewer

whedon commented 4 years ago

The reviewer already has a pending invite.

@athanasiamo please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

drmowinckels commented 4 years ago

jup that did it. Seems like I did not complete the invite when I clicked it the first time. Sorry about the bother.

paulinepalma commented 4 years ago

Hi Olivia,

I think I have the same problem as Athanasia. Could you please send me the invite again? Sorry about that!

Pauline

Le mar. 2 juin 2020, à 09 h 53, whedon notifications@github.com a écrit :

The reviewer already has a pending invite.

@Athanasiamo https://github.com/Athanasiamo please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2285#issuecomment-637558027, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKOWI7KUNVHXYZYPF5SJSBDRUT75VANCNFSM4NQCGMHQ .

-- Sent from my ouija board

danielskatz commented 4 years ago

@paulinepalma - Can you try just accepting the invitation in the first comment in this thread?

oliviaguest commented 4 years ago

@danielskatz thank you so much for this! I should have realised it was that. 😊

danielskatz commented 4 years ago

@oliviaguest - if the invite has expired, the whedon re-invite command can also be used, but in this case, it seems unlike that the invite would have expired :)

paulinepalma commented 4 years ago

Hi Daniel,

That is what I did initially, but here is the message I got: "Sorry, we couldn't find that repository invitation. It is possible that the invitation was revoked or that you are not logged into the invited account." I am logged into my github account.

Pauline

Le mar. 2 juin 2020, à 10 h 27, Daniel S. Katz notifications@github.com a écrit :

@oliviaguest https://github.com/oliviaguest - if the invite has expired, the whedon re-invite command can also be used, but in this case, it seems unlike that the invite would have expired :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2285#issuecomment-637578147, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKOWI7P2UYDIOMEBQH6D2YDRUUD3VANCNFSM4NQCGMHQ .

-- Sent from my ouija board

oliviaguest commented 4 years ago

@whedon re-invite @paulinepalma as reviewer

whedon commented 4 years ago

OK, the reviewer has been re-invited.

@paulinepalma please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

oliviaguest commented 4 years ago

@paulinepalma lemme know if that works, hopefully yes! 🌺

paulinepalma commented 4 years ago

It works! Thank you very much :)

Le mar. 2 juin 2020, à 10 h 58, Olivia Guest notifications@github.com a écrit :

@paulinepalma https://github.com/paulinepalma lemme know if that works, hopefully yes! 🌺

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2285#issuecomment-637598437, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKOWI7IAXQYVYQP4KDBLDUTRUUHS3ANCNFSM4NQCGMHQ .

-- Sent from my ouija board

paulinepalma commented 4 years ago

@oliviaguest I started going through the repository and I am having difficulties installing the developper version of the package (under R version 4.0.0). I ended up installing the CRAN version, which works. It could be an issue with my own computer, so I am unsure whether I should mention this in the issues?

oliviaguest commented 4 years ago

@paulinepalma I think this is exactly the kind of problem/solution @samhforbes should be able to solve for users (and might even already know how to solve). So, yes, please try and discuss with him here. 💯

samhforbes commented 4 years ago

Yes @paulinepalma if you can let me know where it fails, we can hopefully work it out. The development version was made with 4.0.0

oliviaguest commented 4 years ago

Also yes — sorry! — @paulinepalma feel free if you want (if you think it's needed) to create a specific issue for that at the repo. Basically whatever is sensible for you and @samhforbes. As always link to it from here if you do make any related issues. 🌷

paulinepalma commented 4 years ago

Ok! I'll create the issue now.

paulinepalma commented 4 years ago

General checks ✅ Contribution and authorship: Has the submitting author (@samhforbes) made major contributions to the software? Does the full list of paper authors seem appropriate and complete? One contributor (TJ Mahr) made a small commit at some point, should they be in the acknowledgements somewhere? They are also mentioned in the vignette.

Functionality ➖Installation: Does installation proceed as outlined in the documentation? Problem encountered with the developer installation, not with the released CRAN version (https://github.com/samhforbes/PupillometryR/issues/12#issue-629469353). Maybe OS problem on my end? It may be good to specify which version of R and operating system are needed to run the package.

Documentation ➖Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems). Examples from the vignette are clear and detailed. However, with the released version, an error occurs when retrieving the “vignette(‘PupillometryR’)” potentially because the released version does not include the vignette (see https://github.com/samhforbes/PupillometryR/issues/12#issue-629469353) ✅Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support Link to the issues board recently added to the Readme.

Software paper ✅Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)? A few typos, see https://github.com/samhforbes/PupillometryR/issues/13#issue-629496232 ➖References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax? There are a few issues with the references of the paper and of the Readme file, see https://github.com/samhforbes/PupillometryR/issues/14#issue-629499835

oliviaguest commented 4 years ago

@paulinepalma @Athanasiamo thanks for both your reviews so far! Super helpful and I really appreciate the work you are putting in. 👏

I edited your comments. I hope/think I left only the points with relevant info over and above the tick (if I removed anything vital, please feel free to put it back — and sorry if so!). TBC it's to help @samhforbes focus on what he needs to address. The stuff that just has a ✅ is already in the OP. I hope this makes sense!

oliviaguest commented 4 years ago

@samhforbes Just to keep this under control as there are a lot of issues on the code repo, can you close (button at the bottom) these, presuming they are addressed, please? And of course any others that are dealt with. 😊

samhforbes commented 4 years ago

Yes, tidied up now!

samhforbes commented 4 years ago

I'm going to need to update the version because I've had to resubmit to CRAN (archived due to an older dependency that was fixed prior to peer review here).

Please let me know if there's anything else I can do

samhforbes commented 4 years ago

@whedon set v0.0.3 as version

whedon commented 4 years ago

I'm sorry @samhforbes, I'm afraid I can't do that. That's something only editors are allowed to do.

oliviaguest commented 4 years ago

@whedon set v0.0.3 as version

whedon commented 4 years ago

OK. v0.0.3 is the version.

oliviaguest commented 4 years ago

@whedon... you are so very strict! We like you anyway. 😆

whedon commented 4 years ago

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

oliviaguest commented 4 years ago

Hey all! 👋 @paulinepalma, @Athanasiamo, @samhforbes! Thank you to everybody for doing such a great job so far.

👉 Specifically for the reviewers, when they get a chance, check if their issues have been addressed and those that have are checked/ticked off above? 🌸

👉 For Sam, can you let me know which bits you are working on — again when you have the chance, no pressure. 🌼

oliviaguest commented 4 years ago

PS: If all is done, BTW, just let me know so we can start preparing it for publication. 💯

drmowinckels commented 4 years ago

All my points are ticked off :) So I'm happy with this.

samhforbes commented 4 years ago

I've just closed the last issue on the repository, which was to launch a pkgdown website as recommended by @Athanasiamo.

I've also updated the Geller reference as I've seen the paper has now come out in full, and Whedon preview suggests it compiles OK.

oliviaguest commented 4 years ago

Thank you — amazing!

@paulinepalma I assume this is all your requests/ideas addressed, right? 😊

paulinepalma commented 4 years ago

Yes! All of my comments have been addressed.

Le lun. 15 juin 2020, à 12 h 06, Olivia Guest notifications@github.com a écrit :

Thank you — amazing!

@paulinepalma https://github.com/paulinepalma I assume this is all your requests/ideas addressed, right? 😊

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2285#issuecomment-644226216, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKOWI7IQEMYHEPCVQDPYRS3RWZBILANCNFSM4NQCGMHQ .

-- Sent from my ouija board

oliviaguest commented 4 years ago

Wooooo!!!! @paulinepalma @Athanasiamo!!! Thank you both so much for your valuable and timely feedback on improving @samhforbes code and paper! 🎉

oliviaguest commented 4 years ago

@whedon check references

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.12688/wellcomeopenres.15191.1 is OK
- 10.3758/s13428-017-1007-2 is OK
- 10.3758/s13428-018-01190-1 is OK
- 10.3758/s13428-020-01374-8 is OK
- 10.1007/978-1-4757-7107-7 is OK
- 10.1111/j.1467-7687.2008.00805.x is OK
- 10.3758/s13428-011-0109-5 is OK
- 10.1177/2331216519832483 is OK
- 10.3758/s13423-018-1432-y is OK
- 10.1177/2331216518800869 is OK
- 10.1126/science.143.3611.1190 is OK
- 10.1037/h0026952 is OK

MISSING DOIs

- None

INVALID DOIs

- None

oliviaguest commented 4 years ago

@samhforbes can you create an archive (on Zenodo, figshare, or other) and post the archive DOI here, please? 😃

oliviaguest commented 4 years ago

@whedon generate pdf

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

samhforbes commented 4 years ago

Yes thank you both so much @paulinepalma and @Athanasiamo - I'm really grateful for your input, it's a neater and sleeker package for your input.

samhforbes commented 4 years ago

@oliviaguest Up on Zenodo now: 10.5281/zenodo.3897291

oliviaguest commented 4 years ago

@whedon set 10.5281/zenodo.3897291 as archive

openjournals / joss-reviews