search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
708 stars 37 forks source link

[PRE REVIEW]: The SAGE Rejected Article Tracker #2840

Closed whedon closed 3 years ago

whedon commented 3 years ago

Submitting author: @ad48 (Adam Day) Repository: https://github.com/sagepublishing/rejected_article_tracker_pkg Version: 1.5.0 Editor: Pending Reviewer: Pending Managing EiC: Kristen Thyng

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @ad48. Currently, there isn't an JOSS editor assigned to your paper.

The author's suggestion for the handling editor is @lorenanicole.

@ad48 if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands
whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.20 s (194.1 files/s, 8222.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Python                          30            168             61            801
Markdown                         5             91              0            339
TeX                              1              1              0            108
YAML                             1              7              0             41
Bourne Shell                     2              4             21             10
-------------------------------------------------------------------------------
SUM:                            39            271             82           1299
-------------------------------------------------------------------------------

Statistical information for the repository '576d3969bc442182cce7c3ca' was
gathered on 2020/11/13.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Adam Day                         2            15              7            1.67
Andy Hails                      21          1159            137           98.33

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Adam Day                     15          100.0          1.4               26.67
Andy Hails                 1015           87.6          0.4                2.36
whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1002/bjs.6880 is OK
- 10.1007/s00134-018-5407-2 is OK
- 10.3138/W706-5884-KG84-J282 is OK
- 10.1111/anae.13829 is OK
- 10.23876/j.krcp.20.392 is OK

MISSING DOIs

- None

INVALID DOIs

- None
whedon commented 3 years ago

PDF failed to compile for issue #2840 with the following error:

/app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon/author.rb:72:in block in build_affiliation_string': Problem with affiliations for Andrew Hails, perhaps the affiliations index need quoting? (RuntimeError) from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon/author.rb:71:ineach' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon/author.rb:71:in build_affiliation_string' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon/author.rb:17:ininitialize' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon.rb:205:in new' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon.rb:205:inblock in parse_authors' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon.rb:202:in each' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon.rb:202:inparse_authors' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon.rb:93:in initialize' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon/processor.rb:38:innew' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/lib/whedon/processor.rb:38:in set_paper' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/bin/whedon:58:inprepare' from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/command.rb:27:in run' from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:ininvoke_command' from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor.rb:387:in dispatch' from /app/vendor/bundle/ruby/2.6.0/gems/thor-0.20.3/lib/thor/base.rb:466:instart' from /app/vendor/bundle/ruby/2.6.0/bundler/gems/whedon-54ca145bf448/bin/whedon:131:in <top (required)>' from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:inload' from /app/vendor/bundle/ruby/2.6.0/bin/whedon:23:in `

'

kthyng commented 3 years ago

@whedon query scope

whedon commented 3 years ago

Submission flagged for editorial review.

kthyng commented 3 years ago

@ad48 - thanks for your submission! I've flagged this submission to be looked at by the editorial board at large due to its relatively small size to see if it is in scope. Someone will get back to you within a week or two.

danielskatz commented 3 years ago

👋 @ad48 - I'm sorry to say that after discussion amongst the JOSS editors, we have decided that this submission does not meet the substantial scholarly effort criterion for review by JOSS. Please see https://joss.readthedocs.io/en/latest/submitting.html#other-venues-for-reviewing-and-publishing-software-packages for other suggestions for how you might receive credit for your work.

danielskatz commented 3 years ago

@whedon reject

whedon commented 3 years ago

Paper rejected.

ad48 commented 3 years ago

Apologies. We did not receive notifications for your messages above. I can confirm that we did both carefully review the 'substantial scholarly effort' submission criteria and believe that the software meets with these requirements. Is there something specific that was lacking? If permissible, we might retry with a later version if we know what the specific issue was.

danielskatz commented 3 years ago

In my personal opinion, issues are the size of the codebase and the fact that there's no case being made of why this is research software: software intended to be used by researchers to do their research. For example, a citation manager is useful for researchers, but I don't think we would call it research software, as it doesn't help do specific research.

ad48 commented 3 years ago

Thank you for the clarification. The codebase is ~1000 rows which is indeed small but this seems to be in line with JOSS guidelines. It was also recently re-written from another repo - so the stats you have might make it look younger than it is. (But it is intended to be a simple application - not a complex one.) We'd certainly be grateful for criticism if it can be improved.

There are various research papers focused on rejected article tracking, but they have been limited in scope (these are referenced in our submission under 'statement of need'). Part of the reason these projects are so small and so few may be the lack of software: every one of them states that searches were carried out manually. As far as we are aware this is the only open-source automation solution to that research problem.

There are other applications besides those mentioned in the paper.

Would it be possible to re-submit if we clarify these issues and perhaps elaborate on the research applications to make those more clear? The guidelines seemed to emphasise conciseness, but we would be happy to go into greater detail.

danielskatz commented 3 years ago

The current JOSS guidelines is that anything under 1000 LOC is subject to review for just that reason, and this appears to be about 800. In addition, the research software check is a different one. I think what you write there is reasonable. Also note that the paper did not compile (see https://github.com/openjournals/joss-reviews/issues/2840#issuecomment-727085906), which might have made it harder for editors to check on it. You certainty are welcome to resubmit, but I think you will have to add to the software as well.

ad48 commented 3 years ago

Thank you. That's very helpful feedback. I was looking at the total line count for the repo - I see now it's just the code we're interested in.

We'll give it some thought. Thanks again.