search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
721 stars 38 forks source link

[PRE REVIEW]: Synaptus: A Matlab/Octave toolbox for synthetic aperture ultrasound imaging #3579

Closed whedon closed 2 years ago

whedon commented 3 years ago

Submitting author: @mh-skjelvareid (Martin H. Skjelvareid) Repository: https://github.com/mh-skjelvareid/synaptus Version: v1.1.0 Editor: @Kevin-Mattheus-Moerman Reviewers: @Samuel-Wagner, @emanuelhuber Managing EiC: Arfon Smith

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/11a4c24fd2f5b2ff9b7d32206453864f"><img src="https://joss.theoj.org/papers/11a4c24fd2f5b2ff9b7d32206453864f/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/11a4c24fd2f5b2ff9b7d32206453864f/status.svg)](https://joss.theoj.org/papers/11a4c24fd2f5b2ff9b7d32206453864f)

Author instructions

Thanks for submitting your paper to JOSS @mh-skjelvareid. Currently, there isn't an JOSS editor assigned to your paper.

@mh-skjelvareid if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands
whedon commented 3 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf
whedon commented 3 years ago

PDF failed to compile for issue #3579 with the following error: 

 Can't find any papers to compile :-(
whedon commented 3 years ago 
Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.04 s (775.7 files/s, 83697.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
MATLAB                          29            437           1001           1486
TeX                              1             39              3            271
-------------------------------------------------------------------------------
SUM:                            30            476           1004           1757
-------------------------------------------------------------------------------

Statistical information for the repository 'eb7d3bca003125990de97659' was
gathered on 2021/08/06.
No commited files with the specified extensions were found.
arfon commented 3 years ago

@whedon generate pdf from branch paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

arfon commented 3 years ago

:wave: @mh-skjelvareid. Thanks for your submission to JOSS. Could you make some updates to your paper to make the perspective of the writing the same? Towards the end of the paper you write:

Although the toolbox started out as a summary of my PhD work, and I have since started working on other research topics, I would like to keep the toolbox a “live” software project. I am particularly interested in adapting the algorithms in the toolbox to different imaging setups, e.g. different shaped arrays or imaging geometries.

To keep this consistent with the rest of the paper please change this to e.g., Although the toolbox started out as a summary of a PhD program, efforts have since begun focused on other research topics, (hopefully you get the idea).

mh-skjelvareid commented 3 years ago

@arfon - thanks for your feedback! I've changed the paragraph to:

Although the toolbox was first published as a collection of algorithms and datasets developed during a PhD program, the repository is intended to be an open and live development project. Contributions in the form of datasets from new imaging geometries (e.g. differently shaped arrays or layered media) are particularly welcome, as they provide the foundation for developing algorithms for these geometries. However, contributions in the form of code, feature requests or issue reports are all very much appreciated. Given the popularity and availability of Python, a Python implementation of the toolbox algorithms is also seen as a natural continuation of the project.

I hope this is more consistent with the rest of the paper.

mh-skjelvareid commented 3 years ago

@whedon generate pdf from branch paper

whedon commented 3 years ago 
Attempting PDF compilation from custom branch paper. Reticulating splines etc...
whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mh-skjelvareid commented 3 years ago

I've looked through the list of reviewers and found a few with a background that seems to match (matlab/octave, acoustics, signal processing, geophysics) - I'm listing them here without tagging them:

The list is roughly ordered from most to least relevant background (although it's often hard to tell based on the short descriptions).

arfon commented 3 years ago

@mh-skjelvareid – apologies for not communicating this earlier but we're currently managing a large backlog of submissions and the editor most appropriate for your area is already rather busy.

For now, we will need to waitlist this paper and process it as the queue reduces. Thanks for your patience!

Kevin-Mattheus-Moerman commented 3 years ago

@whedon assign me as editor

whedon commented 3 years ago

OK, the editor is @Kevin-Mattheus-Moerman

Kevin-Mattheus-Moerman commented 3 years ago

@arfon I can take this one

Kevin-Mattheus-Moerman commented 3 years ago

@mh-skjelvareid

Here are some recommendations:

As an example, here is a link to one of my projects: https://github.com/gibbonCode/GIBBON, note the use of the README.md, the way a license badge is rendered, the inclusion of a graphical overview image, the linking to contributing guidelines, and a code of conduct. You may also like the way I render "badges" in the readme for the license etc. you can copy the syntax from the readme if you want to use this too.

Quick comments on paper (we may have more later):

Kevin-Mattheus-Moerman commented 3 years ago

@btreeby @dannyramasawmy @jarosjir @thewtex @pharshalp @Scivision @lassoan would you be interested in reviewing this work (a MATLAB/octave toolbox for ultrasound imaging) for the Journal of Open Source Software (JOSS)? The smooth review process takes place on GitHub, and focuses on the software as well as a short paper.

Thanks!

Note under the current COVID19 pandemic we can be quite flexible in terms of giving your time to complete the review. If you are able to review this work in principle but currently lack the time to do so, feel free to indicate a later time when you may or estimate how long it would take to complete.

Note this is a pre-review issue in which reviewers are assigned. Once sufficient reviewers are recruited a dedicated review issue is initiated. In that review issue the reviewers are guided using check lists. Here is an example of a recently completed review issue to give you an idea of what it is like to review for JOSS: https://github.com/openjournals/joss-reviews/issues/2548

lassoan commented 3 years ago

The topic is not exactly in my area of expertise and would require me to invest significant amount of time, which I cannot justify for a GPL-licensed software (that cannot be used freely but may require to buy commercial license for certain uses).

mh-skjelvareid commented 3 years ago

@lassoan I'm quite new to publishing open source software, and I chose a license that seemed to fit, perhaps without knowing all the consequences. It was not my intention that anyone should need to buy commercial licenses to use the software (with Octave being a free alternative to Matlab). If you have suggestions for alternative licenses, I'll be happy to consider them!

lassoan commented 3 years ago

GPL license essentially requires all software that uses your software to be open-source. There may be exceptions depending on how exactly use the GPL code, but it is a complicated discussion that most companies want to entirely avoid. Even if you work at an academic institution, you may want your work make an impact and adopted by companies, or by other academic groups who may want to work with companies.

Apache and MIT are probably the two most popular licenses that do not restrict usage of the software in any way. Apache gives additional peace of mind to users that you don't want to make them use your patented ideas that you can later charge them for.

Unfortunately, Octave itself comes with GPL license, so even if your code has non-restrictive license, many people will still not be able to use it. Python and most Python packages comes without such restrictions.

Kevin-Mattheus-Moerman commented 3 years ago

@btreeby @dannyramasawmy @jarosjir @thewtex @pharshalp @Scivision @lassoan would you be interested in reviewing this work (a MATLAB/octave toolbox for ultrasound imaging) for the Journal of Open Source Software (JOSS)? The smooth review process takes place on GitHub, and focuses on the software as well as a short paper.

Thanks!

Note under the current COVID19 pandemic we can be quite flexible in terms of giving your time to complete the review. If you are able to review this work in principle but currently lack the time to do so, feel free to indicate a later time when you may or estimate how long it would take to complete.

Note this is a pre-review issue in which reviewers are assigned. Once sufficient reviewers are recruited a dedicated review issue is initiated. In that review issue the reviewers are guided using check lists. Here is an example of a recently completed review issue to give you an idea of what it is like to review for JOSS: https://github.com/openjournals/joss-reviews/issues/2548

Kevin-Mattheus-Moerman commented 3 years ago

GPL license essentially requires all software that uses your software to be open-source. There may be exceptions depending on how exactly use the GPL code, but it is a complicated discussion that most companies want to entirely avoid. Even if you work at an academic institution, you may want your work make an impact and adopted by companies, or by other academic groups who may want to work with companies.

Apache and MIT are probably the two most popular licenses that do not restrict usage of the software in any way. Apache gives additional peace of mind to users that you don't want to make them use your patented ideas that you can later charge them for.

Unfortunately, Octave itself comes with GPL license, so even if your code has non-restrictive license, many people will still not be able to use it. Python and most Python packages comes without such restrictions.

@lassoan thanks for your comments on licenses. We respect your comments and understand them, and also that you feel you do not wish to review this work. Of course the license chosen is up to the developers of the software. At JOSS we consider works open source only if they have an OSI approved license.

I see your points on the GPL license. These are valid concerns and certainly GPL can create compatibility issues when using/integrating the work with other software. Interestingly we have also seen reviewers strongly advocate for the GPL license and these had strong feelings against permissive licenses. Proponents of GPL might say for instance that they do not want companies to absorb and commercialize codes and then not release future developments open source. These may also be valid concerns. Either way, as long as it is an OSI approved license the work can be considered for JOSS. As a sidenote, I believe it can be challenging to enforce GPL for MATLAB works, since GPL's "midas touch", as far as I'm aware, would kick in uppon "distribution" of the software. However with MATLAB codes, if one develops codes that use this library one could always simply distribute only ones own code and ask users to download this library as a 3rd party requirement. This circumvents the distribution issue.... Anyway, this is not the place to have an extensive discussion on this. Thanks again for your comments.

Kevin-Mattheus-Moerman commented 3 years ago

@mh-skjelvareid if based the above you have reconsidered your license let us know, again as long as it is an OSI approved open source license your work is compatible with JOSS.

Kevin-Mattheus-Moerman commented 3 years ago

@mh-skjelvareid did you see my remarks above :point_up: (those check boxes), can you formulate a response? FYI you can call @whedon generate pdf here to update the paper.

mh-skjelvareid commented 3 years ago

@Kevin-Mattheus-Moerman @lassoan - excuse me for not replying earlier regarding the license. And thanks, @lassoan, for your insightful comments on licensing options.

To me, the main goal of sharing this open source project is for other researchers to get a working version of a set of algorithms. Having a general, mathematical description in a paper is one thing, but an implementation reveals all the details that there's no room for in the paper. As a reasearcher, I've found that there are hundreds of papers suggesting new/improved algorithms for ultrasound imaging, but that very few authors actually publish code along with their papers. In this way, it's hard to build upon other researchers' work without spending a lot of time making your own implementation - and as a consequece, progress in the field is slow.

If a researcher decides to use one of my algorithms and maybe improve upon it, then it's important to me that the improvements are shared with the rest of the community, and this is why I've chosen the GPL license.

I can see that in some cases, GPL can be restrictive to a company. However, ultrasound imaging is very computationally intensive, and most commercial products use specially designed hardware to accelerate the process. I can't imagine that a company would use Matlab or Octave for this - a more realistic scenario is that they implement the algorithms from scratch (in e.g. C++), based on the original papers and on the general approach outlined by the Matlab/Octave code. To me it seems that this would allow a company to make an implementation without being overly restricted by the GPL license. I might be wrong here - please correct me if I am.

Finally, Python seems like a very good and perhaps more "modern" alternative to Matlab, and I hope to someday make a Python implementation of the algorithms in the toolbox. However, it will require quite a lot of time, which I really don't have at the moment. I feel that publishing a Matlab version and sharing it with the community is better than waiting (potentially a few years) until a Python version is ready.

mh-skjelvareid commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mh-skjelvareid commented 3 years ago

@Kevin-Mattheus-Moerman - thanks for your recommendations! I've followed up on them, with comments for each issue (below).

Here are some recommendations:

  • [x] Change the _readme.txt file to a README.md markdown file. This way GitHub will recognize it and render it on the project front page. This helps folks clearly see what the project is about when they land on your project page.

The README file has been updated to a markdown file.

  • [x] Change the _license.txt file to a plain LICENSE file. This way GitHub will recognize it and render a license badge at the top of your repository. This helps folks instantly see the license type for the project when they land on your page.

The LICENSE file has been updated to a markdown file.

  • [x] Change the _contributing.txt file to a formatted CONTRIBUTING.md file and link to it from the readme.

The CONTRIBUTING file has been updated to a markdown file, and it is linked to from README.

A COC file with updated contact info has been added.

  • [x] If you have a couple of example images that help show the functionality I recommend you render these in your readme file as well as in the paper. You can also consider creating a "graphical abstract" figure that summarizes functionality. This will help folks to visually grasp what the code offers.

I've added a section called "Example raw and focused images" to the README file, with several visual examples of experiment setups, and "before and after" ultrasound data. These images have been taken directly from the PhD thesis distributed with the code, and include figure numbers and captions. This may look a bit odd, but I feel that having a direct reference to the thesis (with more in-depth descriptions of the examples) could be a good thing. However, I'm open to removing the captions if they seem out of place.

I've also included one of the examples in the paper itself. When I originally read the example paper for JOSS, I got the impression that the paper only needed "Summary" and "Statement of need" sections, and didn't think to include example figures. I will let you / the reviewers decide if you think the "Example use" section should be included in the paper or if it seems out of place. The information is included in the README file anyway, and I wasn't sure if I should duplicate it in the paper.

I haven't created a graphical abstract, but I think the example(s) are quite visual, and I think they may be enough in themselves.

As an example, here is a link to one of my projects: https://github.com/gibbonCode/GIBBON, note the use of the README.md, the way a license badge is rendered, the inclusion of a graphical overview image, the linking to contributing guidelines, and a code of conduct. You may also like the way I render "badges" in the readme for the license etc. you can copy the syntax from the readme if you want to use this too.

The README file/page for GIBBON both looks nice and is very well structured - thanks for pointing me to it, it's a good example to follow. I've copied the use of a license badge in the README file.

Quick comments on paper (we may have more later):

  • [x] In “raw data,” it looks like the comma should be outside of the quotation marks e.g. “raw data”,
  • [x] Check typo: ultrasund -> ultrasound

Fixed.

Kevin-Mattheus-Moerman commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

Kevin-Mattheus-Moerman commented 3 years ago

@whedon check references

whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1190/1.1440826 is OK
- 10.1190/1.1440899 is OK
- 10.1017/9781316756041 is OK
- 10.1109/TUFFC.2010.1718 is OK
- 10.1109/TUFFC.2007.400 is OK
- 10.1109/TUFFC.2011.1904 is OK
- 10.1109/TUFFC.2012.2478 is OK
- 10.1063/1.3703163 is OK
- 10.1016/j.ndteint.2012.10.005 is OK
- 10.1063/1.4928118 is OK
- 10.1109/ULTSYM.2017.8092389 is OK

MISSING DOIs

- 10.1002/(issn)1873-0604 may be a valid DOI for title: Near Surface Geophysics

INVALID DOIs

- None
Kevin-Mattheus-Moerman commented 3 years ago

@mh-skjelvareid thanks. Can you also work on that potentially missing DOI? Also it looks like the “raw data,” issue is still in there.

mh-skjelvareid commented 3 years ago

@whedon check references

whedon commented 3 years ago 
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1190/1.1440826 is OK
- 10.1190/1.1440899 is OK
- 10.1017/9781316756041 is OK
- 10.1109/TUFFC.2010.1718 is OK
- 10.1109/TUFFC.2007.400 is OK
- 10.1109/TUFFC.2011.1904 is OK
- 10.1109/TUFFC.2012.2478 is OK
- 10.1063/1.3703163 is OK
- 10.1016/j.ndteint.2012.10.005 is OK
- 10.1063/1.4928118 is OK
- 10.1109/ULTSYM.2017.8092389 is OK

MISSING DOIs

- None

INVALID DOIs

- None
mh-skjelvareid commented 3 years ago

@Kevin-Mattheus-Moerman , I've fixed the missing DOI (the reference was to a Github repository that doesn't have a DOI, but the name of the repository is similar to another publication).

That "raw data," issue is a real mystery, however. If you check the markdown file for the paper, it's correct (i.e., "raw data",) while in the compiled PDF it changes to "raw data," . A LaTeX issue?

Kevin-Mattheus-Moerman commented 3 years ago

@openjournals/dev can you see what is wrong with this :point_up:

arfon commented 3 years ago

Wow, that's deeply peculiar! No idea. I even tried writing this out as &quot;raw data&quot;&#44; in the paper.md but to no avail. I suggest we don't get blocked on this for now, but perhaps @tarleb has some ideas?

Kevin-Mattheus-Moerman commented 3 years ago

@arfon Could we add an additional space to force it?

tarleb commented 3 years ago

Could you try setting

---
lang: en-GB
---

in the YAML metadata block to force the comma out of the quote?

arfon commented 3 years ago

@mh-skjelvareid – could you merge this PR so we can try @tarleb's suggestion? https://github.com/mh-skjelvareid/synaptus/pull/4

mh-skjelvareid commented 3 years ago

@arfon Done!

arfon commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

PDF failed to compile for issue #3579 with the following error: 

 Error producing PDF.
! LaTeX Error: File `polyglossia.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 
! Emergency stop.
<read *> 

l.251   \setmainlanguage

Looks like we failed to compile the PDF
arfon commented 3 years ago

😢 ok, looks like that's not a quick fix. I suggest reverting that change @mh-skjelvareid and we'll make sure we address this before we publish.

mh-skjelvareid commented 3 years ago

@whedon generate pdf

whedon commented 3 years ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

mh-skjelvareid commented 3 years ago

Thanks for trying, @arfon ! I've reverted the change, and also decided to remove the quotation marks altogether - they're not really required in the context.

Kevin-Mattheus-Moerman commented 3 years ago

@btreeby @dannyramasawmy @jarosjir @thewtex @pharshalp @Scivision @thewtex @pharshalp @AnantHariharan1996 @benjaminbolling @mlp6 @tboerstad would you be interested in reviewing this work (a MATLAB/octave toolbox for ultrasound imaging) for the Journal of Open Source Software (JOSS)? The smooth review process takes place on GitHub, and focuses on the software as well as a short paper.

Thanks!

Note under the current COVID19 pandemic we can be quite flexible in terms of giving your time to complete the review. If you are able to review this work in principle but currently lack the time to do so, feel free to indicate a later time when you may or estimate how long it would take to complete.

Note this is a pre-review issue in which reviewers are assigned. Once sufficient reviewers are recruited a dedicated review issue is initiated. In that review issue the reviewers are guided using check lists. Here is an example of a recently completed review issue to give you an idea of what it is like to review for JOSS: https://github.com/openjournals/joss-reviews/issues/2548