Closed whedon closed 2 years ago
Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.
:warning: JOSS reduced service mode :warning:
Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.
For a list of things I can do to help you, just type:
@whedon commands
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
@whedon generate pdf
Wordcount for paper.md
is 732
Software report (experimental):
github.com/AlDanial/cloc v 1.88 T=0.26 s (216.3 files/s, 76470.1 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
JSON 3 0 0 16768
Python 22 365 169 821
JavaScript 8 84 15 579
YAML 5 42 356 412
Markdown 4 67 0 149
TeX 1 11 0 89
CSS 3 8 0 56
HTML 2 3 20 37
Dockerfile 5 14 10 32
Bourne Shell 3 13 0 28
SVG 1 0 0 7
-------------------------------------------------------------------------------
SUM: 57 607 570 18978
-------------------------------------------------------------------------------
Statistical information for the repository '06e55491a54eb028252e02ad' was
gathered on 2022/01/21.
The following historical commit information, by author, was found:
Author Commits Insertions Deletions % of changes
Jorgen Ader 14 104 93 0.04
Jyrno Ader 4 17 1 0.00
Maged Helmy 323 257727 255721 99.96
Below are the number of rows from each author that have survived and are still
intact in the current revision:
Author Rows Stability Age % in comments
Jorgen Ader 27 26.0 1.1 18.52
Jyrno Ader 16 94.1 0.4 0.00
Maged Helmy 1990 0.8 1.5 7.14
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):
OK DOIs
- None
MISSING DOIs
- 10.1007/s00134-018-5070-7 may be a valid DOI for title: Second consensus on the assessment of sublingual microcirculation in critically ill patients: results from a task force of the European Society of Intensive Care Medicine
- 10.1109/ms.2021.3089730 may be a valid DOI for title: Database of the Year: Postgres
INVALID DOIs
- None
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
Hello @magedhelmy1, thanks for your interest in JOSS.
I noticed that you described your submission as an "original research study". JOSS does not actually review or publish original research, but instead research software packages: https://joss.readthedocs.io/en/latest/submitting.html
Original research should be submitted to a domain journal for review; we would review software that was developed in support of or related to that research. Can you clarify how your submission meets that requirement?
Hi @kyleniemeyer , thanks for your comment.
Indeed, this is not an original research study to be published here. But this is a research software package intended to address a research challenge: quantifying blood capillaries in patients. This software research package aims to support the functioning of research instruments (blood vessel quantification) and the execution of research experiments (quantifying the capillaries in the skin). My apologies if that was not clear.
Another small typo is in the title "HDMIA: An Architect for High Demand Medical Image 2 Analysis using Deep Learning", the number "2" should not exist there. Again, please accept my apologies for that.
Hi @kyleniemeyer , I hope you had a good weekend.
In addition to the previous answer, I include a detailed answer where applicable below to the points in the link shared.
[X] The software must be open source as per the OSI definition.
[X] The software must have an obvious research application.
The research application of the proposed research software is the ability to quantify capillary density in a microcirculation image in under 2 seconds. Capillaries are the smallest vessels in the body responsible for delivering oxygen and nutrients to surrounding cells. Various life-threatening diseases alter the density of healthy capillaries within the capillaries. In previous studies, capillary density was manually assessed by trained specialists. However, manual analysis of a standard 20-second microvascular video requires 2 minutes on average and necessitates extensive training. Thus, manual analysis of capillary density has been reported to hinder the application of microvascular microscopy in a clinical environment. This paper presents research software to quantify the capillaries to address this problem.
[X] You must be a major contributor to the software you are submitting, and have a GitHub account to participate in the review process.
I, the main author, have 99.96% contribution to the code as per the commits.
No new research has been accomplished in this paper. The proposed research software quantifies the capillaries to enable doctors and clinical researchers to develop their medical hypotheses with regard to specific diseases and capillary density alternation.
This paper solves a problem in medicine, specifically the automated quantification of capillaries in Microcirculation images.
First, we enable some new research challenges to be addressed by allowing the complete automation of quantifying the capillary density in Microcirculation images. On the second fold, the proposed software makes addressing research challenges significantly better by enabling doctors, researchers, and clinical personnel to develop their hypotheses with regards to the significance of capillary density changes in patients with specific diseases. More concretely, it allows them to assess if various diseases indeed alter the capillary density and by how much.
Almost daily commits for the past 120 days (with the exclusion of Christmas and new year, weekends and public holidays)
[X] Age of software (is this a well-established software project) / length of commit history.
[X] Number of commits.
660 commits
[X] Number of authors.
Me plus two professors in computer science and one associate professor in mathematics since we spent 1-year surveying state of the art and preparing the proposed architecture in this research package. This can be evident that the capillary quantification algorithm proposed in JOSS can be found in a time-stamped paper on Arxix from 23 Apr 2021
[X] Total lines of code (LOC). Submissions under 1000 LOC will usually be flagged, those under 300 LOC will be desk rejected.
According to the experimental software report 18978
[ ] Whether the software has already been cited in academic papers.
A different set of authors has informed me they are working on 2 separate papers that will use the proposed research software. The first paper will compare the alteration of capillary density between recovering COVID19 patients and dying COVID19 patients. The second paper will compare the capillary density between pigs with a mechanical heart and pigs with a normal heart. Another set of authors has informed me that they will explore the option of using the proposed software to compare the capillary density in healthy subjects and patients with diabetes and if capillary density can be used to predict amputation of a diabetic patient.
Both papers are still in writing with the authors.
Our proposed software is currently used by a company that provides microcirculation analysis software. Furthermore, the authors mentioned in the previous point have cited our software and those citations will be public if their papers have been accepted for publication. We are optimistic that more will cite and use our proposed research package since it automates a manual process with comparable accuracy to the manual researcher.
As part of the repo, we deployed our system here http://www.analysecapillary.space/ so it can be used out of the box.
As indicated in the README, the details of the capillary detection and quantification algorithm can be found here CapillaryNet: An Automated System to Quantify Skin Capillary Density and Red Blood Cell Velocity from Handheld Vital Microscopy. The paper is currently under review with the Artificial Intelligence in Medicine journal (AIIM). However, the CapillaryNet paper submitted to AIIM covers more algorithms than just capillary detection and capillary quantification (Capillary Velocity, Intra mediate flow and Capillary Flow Direction). The submitted research software to JOSS focuses on capillary quantification and in addition, its deployment, usage, software architecture, and the "High Demand Part" where the servers have an RPS of 3000+.
@whedon invite @Bisaloo as editor
:wave: @Bisaloo – would you be able to edit this submission for JOSS?
@Bisaloo has been invited to edit this submission.
:wave: :wave: @arfon @magedhelmy1, yes, I can edit this submission
@whedon assign me as editor
OK, the editor is @Bisaloo
Hi @magedhelmy1 :wave:, I'll be your topic editor for this submission.
The first step in this pre-review is to find at least 2 reviewers. If you have any suggestions for potential reviewers, please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).
Hi @arfon, Thanks for assigning a reviewer to this. Hi @Bisaloo, glad you are on board to be the topic editor for this submission.
I went through the Google sheet and can suggest several names. Any 2 who are available of the following as you see fit.
sgk98 arn1291 gxlarson AnthonyOfSeattle Ykarpate nnadeau dirmeier vyasr lucaferranti rabdill tuelwer robbisg erik-whiting
Thanks for the suggestions! I'm going to already request some changes, as to facilitate the life of potential reviewers and already clarify certain points about this submission. Could you please:
ter temp
file should not be here. There might be others, I recommend you double check the entire repo for such files.LICENSE
by the actual text of the CC BY-NC 4.0 license, so it can be auto-detected by GitHub and by other toolsSECURITY.md
since it doesn't seem that you updated the boilerplate text and it doesn't mean anything in the context of your repository.contributing.md
file is currently empty. This contributing guide should detail, among other things, how to report an issue (what kind of information should be provided), explain which files should/can be changed by contributors vs files that are auto-generated (you should then also explain how they are auto-generated), how to test your changes (possibly by running automated tests if you have any), and provide at least an example file.Hi @Bisaloo, thank you for being proactive and listing the improvements! I appreciate it. I have incorporated all but one. Below are my comments on each point you stated and the corresponding change location. I am only stuck at one point, which is the second point. Please clarify it, and I will improve right away.
I have updated the references and added DOIs where applicable. The changes are reflected in paper.bib I am unsure how to re-run Whedon to see the report generated and fix the consequential issues.
Extended figures caption to include more details
ter temp
file should not be here. There might be others, I recommend you double check the entire repo for such files.I have now removed
ter temp
,jq.exe
,ray
and other similar files and folders that are obolsete.
LICENSE
by the actual text of the CC BY-NC 4.0 license, so it can be auto-detected by GitHub and by other toolsI have added this, and the changes are reflected in license.md
SECURITY.md
since it doesn't seem that you updated the boilerplate text, and it doesn't mean anything in the context of your repository.I have now removed that file,
SECURITY.md
.
contributing.md
file is currently empty. This contributing guide should detail, among other things, how to report an issue (what kind of information should be provided), explain which files should/can be changed by contributors vs files that are auto-generated (you should then also explain how they are auto-generated), how to test your changes (possibly by running automated tests if you have any), and provide at least an example file.I have updated this and the changes are reflected in README.MD and contributing.md
I have now added that section, and the changes are reflected in project structure
For the caption, I meant a more informative & more descriptive caption. At the moment, as a reader, I don't really understand what I'm looking at and what information I should get from these figures.
Hi @tuelwer :wave:, I see that you volunteered to review for JOSS and this submission seems to fit perfectly your expertise, both in terms of topic and programming language.
Would you be available to review this submission?
Please note that this is a somewhat complex submission with several layers. If you want to review only one of these layers (e.g., the python backend but not the react frontend), this is perfectly fine :slightly_smiling_face:
Thank you @Bisaloo for taking the time to clarify. I have now extended the captions of the figures to include your feedback. Please let me know if I can improve it further or if you have other questions.
@whedon generate pdf
:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:
This is very interesting work!
@Bisaloo unfortunately, I am currently busy and cannot review this submission in reasonable time. Please, feel free to contact me again in a couple of weeks if you have not found a reviewer by then!
@tuelwer thank you! and thanks for responding fast.
Thanks for your quick answer @tuelwer, I'll ping you again in a couple of weeks if necessary :slightly_smiling_face:
Hey @Bisaloo , happy to suggest other names if needed besides the 13 above.
Hi @fepegar :wave:, I see you volunteered to review submissions for JOSS. This submission, with the associated pape, seems to fit quite well with your expertise. Would you be able to review it?
Please note that this is a somewhat complex submission with several layers. If you want to review only one of these layers (e.g., the python backend but not the react frontend), this is perfectly fine :slightly_smiling_face:
Hi @michaelberks :wave:, I see you volunteered to review submissions for JOSS. This submission, with the associated paper, seems to fit quite well with your expertise. Would you be able to review it?
Please note that this is a somewhat complex submission with several layers. If you want to review only one of these layers (e.g., the python backend but not the react frontend), this is perfectly fine :slightly_smiling_face:
Hi @fepegar 👋, I see you volunteered to review submissions for JOSS. This submission, with the associated pape, seems to fit quite well with your expertise. Would you be able to review it?
Please note that this is a somewhat complex submission with several layers. If you want to review only one of these layers (e.g., the python backend but not the react frontend), this is perfectly fine 🙂
Hi, @Bisaloo. Thanks for pinging me. Unfortunately, this topic is not really close to my areas of knowledge. Good luck finding someone!
Hi @robbisg :wave:, the author has identified you as a potential good reviewer for this submission.
Would you be available to review this piece of software and associated paper for JOSS?
Kind reminder to Bisaloo comments to check if you are available to review the paper submitted to JOSS :)
@michaelberks @robbisg
Sorry for the delay! Yes I'm available!
Thanks for your answer @robbisg! :pray: Please see my comment below.
@magedhelmy1, while I was looking for reviewers, some comments have been raised about whether this submission is really in scope for JOSS. In particular:
Please let me some time to gather the opinions of other editors, and I'll get back to you as soon as possible. Thanks
@whedon query scope
Submission flagged for editorial review.
While useful for many areas of research, pre-trained machine learning models and notebooks are not in-scope for JOSS.
Hi @Bisaloo ,
at a first sight it seemed to me that this software is a platform for uploading images and applying pre-trained methods for the identification of some diseases. Despite it can be useful for some research activity since it detects capillarity density from images, speeding up this process for clinical applications, it seems that is a bit out of the scope of JOSS, as far as I understood and as pointed out by @danielskatz .
Hi @danielskatz , thanks for the hyperlink and highlight. This paper is not a pre-trained machine learning model. The focus of this paper is not the algorithm but rather the "Architecture for High Demand Medical Image Analysis..." built as a platform powered by deep learning models. The algorithm is added for sake of completion and usability of the proposed architecture.
In Figure 3 in the paper, you will see where the "pre-trained" model is in relation to the whole system. It barely occupies 5% of the total system workflow. In fact, it occupies around 2 lines of code in the whole project.
The pre-trained algorithm is submitted as a separate paper for publication in another journal including the data used for training it. This paper focuses on the deployment of this model by coupling several systems together. This way my aim is to meet the following requirement from JOSS The software must have an obvious research application.
is @robbisg comment which is it detects capillarity density from images, speeding up this process for clinical applications,
or else this is an architecture was no obvious research use.
@robbisg There are several JOSS papers that provide software platforms that do improve existing research activities as you highlighted yourself, for example:
https://joss.theoj.org/papers/10.21105/joss.00040
You can find a few more here if you search https://joss.theoj.org/papers/published
With regards to the JOSS requirements, I have a detailed list earlier https://github.com/openjournals/joss-reviews/issues/4087#issuecomment-1020590129 , I'd appreciate it if you highlight there which part is a bit out of the scope of JOSS and I am happy to address.
Thanks @magedhelmy1 - the editors are/will examine this and make a decision, and your comments will be helpful in that process
Thank you @danielskatz for keeping me posted. I appreciate it.
Since the comments might be helpful, I will take this opportunity to also address @Bisaloo comments.
it seems to be more a clinical/medical tool than a research tool when our policy states: "The software must have an obvious research application."
This is a clinical tool, yes, but that does not mean it is not a research tool also. In fact, I did go the extra step and built the clinical tool on top of the architecture to demonstrate the usefulness of the proposed architecture as a research tool. In this case, analyze capillary density in under 2 seconds. This should also satisfy the sentence in that same document later which states
Whether or not the software is sufficiently useful that it is likely to be cited by your peer group.
This is a general-use tool that can be adapted to many similar types of research because it provides a general architecture for pushing images through a deep learning model to provide results in real-time.
Thus, I would argue there are 2 interlinked obvious research applications and they are
1) Researchers who are interested in Microcirculation analyses can simply build on this architecture to analyze many different kinds of Microcirculation images. Some expansions on this package are:
I will not state them all here, but if you have the time, please feel free to check table 9 in Second consensus on the assessment of sublingual microcirculation in critically ill patients: results from a task force of the European Society of Intensive Care Medicine where researchers will need such tools if they will to use microcirculation analysis in bedside inside hospitals.
2) As the title captures, this is an architect on how to analyze medical images using deep learning. So if one would strip away the "tool" part from the code, that is the microcirculation images and deep learning code. You can use this architecture to analyze any image in high demand using any deep learning model. In the performance testing, I show that you are able to get up to 3000 Response per Second which is the equivalent of 10.8million RPS per hour. Thus, the "High Demand" part is in the title.
it is not clear to me how a trained ML model without the training data fits in our scope
As highlighted in the earlier comment that this paper is not about the trained ML Model thus I did not provide the data. A paper with the model+data by the same authors is under consideration in another journal (AIIM). I am happy to modify the paper to reflect the above statements since it seems I did not do that adequately.
Happy to address any further comments or clarify myself further.
hey @danielskatz, I hope all is well. Any news?
👋 @magedhelmy1 - I'm sorry to say that after discussion amongst the JOSS editors, we have decided that this submission has issues in both the JOSS research software and substantial scholarly effort criteria.
This does not mean that it is not software that is useful in research, but just that JOSS does not consider it in scope for review as a substantial contribution of research software. Please see https://joss.readthedocs.io/en/latest/submitting.html#other-venues-for-reviewing-and-publishing-software-packages for other suggestions for how you might receive credit for your work.
Our specific comments are that the submission seems mostly targeted to clinicians, as clearly framed in the paper. This is also highlighted by the fact that there is no way to analyse a folder of images in bulk, or even to export the results of the analysis in a data frame. Currently, the user has to copy the results manually to aggregate them. Additionally, it is not clear what this submission is really about. One major part seems to be a pre-trained ML model, which would make it outside of JOSS scope (+ this part is currently under consideration for publication in another journal according to the author). You defended your submission by saying that the pre-trained ML model is just here to illustrate their submission but that the actual object of interest here is the "architecture", i.e., the fact that you built a react frontend to this ML model and a system to handle the communication between the frontend and the backend. But if we ignore the ML model, we don't think this qualifies as a substantial scholarly effort, as illustrated by the very low number of lines of code.
@whedon reject
Paper rejected.
Submitting author: @magedhelmy1 (Maged Helmy) Repository: https://github.com/magedhelmy1/capillarydetection Version: 0.1 Editor: @Bisaloo Reviewer: Pending Managing EiC: Kyle Niemeyer
:warning: JOSS reduced service mode :warning:
Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.
Status
Status badge code:
Author instructions
Thanks for submitting your paper to JOSS @magedhelmy1. Currently, there isn't an JOSS editor assigned to your paper.
The author's suggestion for the handling editor is @Bisaloo.
@magedhelmy1 if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).
Editor instructions
The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type: