[REVIEW]: An open source crash course on parameter estimation of computational models using a Bayesian optimization approach

whedon commented 4 years ago

Submitting author: @mbarzegary (Mojtaba Barzegari) Repository: https://github.com/mbarzegary/educational-bayesian Version: v2.0 Editor: @magsol Reviewers: @jcapriot, @arokem Archive: 10.5281/zenodo.5013865

Status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/a183b3613fc340277f964d1dd2de7814"><img src="https://jose.theoj.org/papers/a183b3613fc340277f964d1dd2de7814/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/a183b3613fc340277f964d1dd2de7814/status.svg)](https://jose.theoj.org/papers/a183b3613fc340277f964d1dd2de7814)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@e-dub & @jcapriot, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @magsol know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @e-dub

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSE code of conduct.

General checks

[x] Repository: Is the source for this learning module available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
[ ] Version: v2.0
[ ] Authorship: Has the submitting author (@mbarzegary) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

[ ] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
[x] Installation instructions: Is there a clearly stated list of dependencies?
[ ] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

[x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
[x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
[ ] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
[ ] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
[ ] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[ ] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
[ ] Description: Does the paper describe the learning materials and sequence?
[ ] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
[ ] Could someone else teach with this module, given the right expertise?
[ ] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
[ ] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

Review checklist for @jcapriot

Conflict of interest

[x] As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSE code of conduct.

General checks

[x] Repository: Is the source for this learning module available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
[x] Version: v2.0
[x] Authorship: Has the submitting author (@mbarzegary) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

[x] A statement of need: Do the authors clearly state the need for this module and who the target audience is?
[x] Installation instructions: Is there a clearly stated list of dependencies?
[x] Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

[x] Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
[x] Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
[x] Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
[x] Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
[x] Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

[x] Authors: Does the paper.md file include a list of authors with their affiliations?
[x] A statement of need: Does the paper clearly state the need for this module and who the target audience is?
[x] Description: Does the paper describe the learning materials and sequence?
[x] Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
[x] Could someone else teach with this module, given the right expertise?
[x] Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
[x] References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

whedon commented 4 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @e-dub, @jcapriot it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/jose-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/jose-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon commented 4 years ago

Reference check summary:

OK DOIs

- 10.1137/1.9780898717921 is OK
- 10.1007/978-94-009-0909-0 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon commented 4 years ago

:point_right: Check article proof :page_facing_up: :point_left:

magsol commented 4 years ago

Hey @e-dub and @jcapriot , wanted to check in and see how things were going. Any kind of ETA or obstacles I can help with?

magsol commented 4 years ago

Hey @e-dub and @jcapriot, please advise if you're running into any blockers that I can help knock down. Would love to get the reviews for this in soon.

jcapriot commented 4 years ago

@magsol Sorry for the lack of response, I will get this today and work through it. It looks like the original accept link has expired for me though.

magsol commented 4 years ago

Hi @e-dub and @jcapriot , how are things going? Anything I can help with?

magsol commented 4 years ago

Hi @jcapriot:

Any word on when you could finish the review? I'd really like to get this moving.

mbarzegary commented 3 years ago

@magsol may I start to answer the comments? or I should wait for the second review to come. it's almost one year since I started this process.

magsol commented 3 years ago

@mbarzegary Yes, please respond to the comments! I had been planning to follow up on this thread and see if you even wanted the review to go forward since I hadn't heard anything from you responding to existing feedback. If you want to continue, then yes--please respond to the feedback already given, and I will track down another reviewer if @e-dub does not respond ASAP.

e-dub commented 3 years ago

@mbarzegary Yes, please respond to the comments! I had been planning to follow up on this thread and see if you even wanted the review to go forward since I hadn't heard anything from you responding to existing feedback. If you want to continue, then yes--please respond to the feedback already given, and I will track down another reviewer if @e-dub does not respond ASAP.

Respond??? See here

magsol commented 3 years ago

@e-dub I'm so sorry! I'd blame it on "dad brain" (I'm on paternity leave at the moment so everything is super fuzzy) but I should have just scrolled up a bit further; my apologies.

@mbarzegary Please check out the "issues" tab for your submission, under which @e-dub has submitted numerous pieces of feedback you can begin to address.

@magsol Sorry for the lack of response, I will get this today and work through it. It looks like the original accept link has expired for me though.

@jcapriot Here is the link https://github.com/mbarzegary/educational-bayesian/

magsol commented 3 years ago

Hi folks. Haven't seen any movement here since the last post a couple weeks ago.

@jcapriot Please go ahead and use the link in the previous post to review the submission.

@mbarzegary Please start reviewing the feedback provided by @e-dub (https://github.com/mbarzegary/educational-bayesian/issues) and addressing it. This will speed up review.

mbarzegary commented 3 years ago

Hi all. Thank you for the valuable feedback. I addressed the review comments.

magsol commented 3 years ago

@e-dub: how does the feedback look?

@jcapriot Any movement on your review?

jcapriot commented 3 years ago

I'm moving on it now, again I'm very sorry for my delay in getting through this as my life has been in a constant flux recently. Again, I will need the link to accept the invitation, as that seems to have expired for me. (Not the link to the repository) As I cannot edit the checklist on this page.

magsol commented 3 years ago

@jcapriot No worries; 2020 has been a barnburner for all of us in many different ways... to put it lightly.

Hmm; I'm relatively new as an editor so this is the first time I've seen this problem. From what I can tell, it looks like you need to be 1) logged in (clearly you are), and 2) accept the invite at this link: https://github.com/openjournals/joss-reviews/invitations .

If that still doesn't solve the problem, let me know and I'll start pinging people up the chain of command.

jcapriot commented 3 years ago

I keep getting this error when clicking the link.

magsol commented 3 years ago

Hi @openjournals/joss-eics:

We're running into a problem where one of the reviewers can't seem to modify the review checklist in this ticket. I see @jcapriot 's account listed as "Awaiting jcapriot's response" under the "Manage Access" page for jose-reviews repository, but you can see from the above post that he's not having any success accepting the invitation.

I'm out of ideas. Could you provide some assistance?

kthyng commented 3 years ago

Hm I didn't seem @jcapriot for JOSS access. I'll invite again just in case that fixes it. @jcapriot if you have more than one github account, make sure you are using the correct login.

kthyng commented 3 years ago

@whedon re-invite @jcapriot as reviewer

whedon commented 3 years ago

I'm sorry @kthyng, I'm afraid I can't do that. That's something only editors are allowed to do.

kthyng commented 3 years ago

Oh! This is JOSE not JOSS! You need to ping @openjournals/jose-eics instead.

magsol commented 3 years ago

@kthyng Oh geez, I'm sorry about that. Autocorrect for the loss :P

@openjournals/jose-eics , please see this message above (ignore that I accidentally tagged joss-eics).

magsol commented 3 years ago

@jcapriot I was able to cancel and resubmit your invitation to modify the JOSE GitHub repos. Let me know if this works.

magsol commented 3 years ago

@e-dub Any suggestions on the feedback provided by @mbarzegary ?

magsol commented 3 years ago

Hi all 👋

The reviews are going silent again. Please respond if you want to continue the review process, otherwise we'll have to end review without acceptance.

@e-dub: Can you look over the responses to your initial feedback provided by @mbarzegary and verify that they satisfy your initial concerns?

@jcapriot: Please check for an invitation to modify the JOSE GitHub repos again, as I canceled and resubmitted your request 12 days ago. Notify me ASAP if you still cannot perform edits; otherwise, please proceed with your review.

mbarzegary commented 3 years ago

a rejection after 10 months is very disappointing for me :(, but apparently this is the dark side of the "community review". @magsol may I resubmit the paper with you again as the editor?

labarba commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

labarba commented 3 years ago

@whedon check repository

whedon commented 3 years ago

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.07 s (83.9 files/s, 18008.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Jupyter Notebook                 1              0            828            233
Markdown                         3             44              0             88
TeX                              1              6              0             60
Python                           1              7              6             16
-------------------------------------------------------------------------------
SUM:                             6             57            834            397
-------------------------------------------------------------------------------

Statistical information for the repository '4431d99643a38fd4b42f4759' was
gathered on 2021/02/21.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
mbarzegary                       1            29              0          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
mbarzegary                   29          100.0          0.0               20.69

labarba commented 3 years ago

@mbarzegary — Can you confirm here that you have addressed all the comments of the first reviewer (@e-dub) and are ready for a second look? (I have reached out by email.)

labarba commented 3 years ago

Hi @jcapriot — I see that you opened some issues in the target repository back in December. Can you check of items in your review checklist now? Are the permission problems solved? Let us know if you can give this submission another look now.

mbarzegary commented 3 years ago

dear @labarba yes, I confirm that I have addressed them all. I reopened all the issues to be closed by @e-dub after confirmation.

labarba commented 3 years ago

Hi @e-dub 👋 — the author says they have addressed all the issues you opened in the target repository. Could you give this a second look now, and see if you can check off (some of) the remaining items in your checklist?

labarba commented 3 years ago

@magsol — I have searched quite a bit, but could not find an email for @jcapriot... do you have one? I found in his LinkedIn profile that he is now a postdoc at UBC, after graduating last year. If he does not respond anymore, we will need a replacement reviewer. Please have a look around...

jcapriot commented 3 years ago

@labarba I still don't have the ability to edit my boxes on this page, and I've never actually received the link to be able to join the JOSE repo to edit them. It still says that I do not have permissions, and I have tried many times to gain permissions. I have not seen it come through. I should have my email listed on my GitHub account here. I did add comments and errors in a couple issues on the target repo that look to have been addressed.

labarba commented 3 years ago

@whedon re-invite @jcapriot as reviewer

whedon commented 3 years ago

OK, the reviewer has been re-invited.

@jcapriot please accept the invite by clicking this link: https://github.com/openjournals/jose-reviews/invitations

labarba commented 3 years ago

hey @jcapriot 👋 Nice to see you pop in here! Try the link ☝️

jcapriot commented 3 years ago

Thank you, sorry for all of the confusion :/

magsol commented 3 years ago

Thanks @labarba!

@jcapriot Let me know if you need anything else to complete your review.

@e-dub Please follow up regarding whether @mbarzegary has adequately addressed your feedback, or if anything else is needed. I'd like to finalize your review as soon as possible.

magsol commented 3 years ago

Hi folks,

We really need to move on this.

@jcapriot: Please let me know when you can complete your review. If it's finished, please let me know so we can finalize your end.

@e-dub Please follow up on whether @mbarzegary has adequately addressed your reviewer feedback. If so, I'd like to finalize your portion of the review.

Let's get this done as soon as possible. Thank you.

magsol commented 3 years ago

@mbarzegary We're between a bit of a rock and a hard place. Are you waiting on either of the reviewers at this point?

mbarzegary commented 3 years ago

@magsol technically speaking, no. the second reviewer has left some of the checkboxes unchecked without creating issues in the repository, so I don't know his comments on the items to fix them. I have fixed most of the unchecked items according to the comments of reviewer 1.

magsol commented 3 years ago

Hi @jcapriot 👋 Please see the note from the author below regarding your review checklist:

@magsol technically speaking, no. the second reviewer has left some of the checkboxes unchecked without creating issues in the repository, so I don't know his comments on the items to fix them. I have fixed most of the unchecked items according to the comments of reviewer 1.

Please respond as soon as you can with whatever help you may need from me or the author, and whatever additional steps you have yet to take.

magsol commented 3 years ago

@jcapriot If you cannot complete your review, please advise ASAP so we can find another reviewer.

jcapriot commented 3 years ago

Hello everyone, I have gone through and updated the checkboxes. There is still an unresolved issue related to my review on the target repo, (from Dec 10th: https://github.com/mbarzegary/educational-bayesian/issues/20) that I feel is the only place where this module is significantly lacking.

magsol commented 3 years ago

Thank you very much @jcapriot!

@mbarzegary Please look through the feedback and see if you can address the issue.

openjournals / jose-reviews