search

openjournals / joss-reviews

Reviews for the Journal of Open Source Software
Creative Commons Zero v1.0 Universal
715 stars 38 forks source link

[REVIEW]: DynaMo: Dynamic Body Shape and Motion Capture with Intel RealSense Cameras #1466

Closed whedon closed 5 years ago

whedon commented 5 years ago

Submitting author: @abhishektha (Abhishektha Boppana) Repository: https://github.com/anderson-cu-bioastronautics/dynamo_realsense-capture Version: v1.1 Editor: @labarba Reviewers: @melund, @ixjlyons, @alcantarar Archive: 10.5281/zenodo.3464497

Status

status

Status badge code:

HTML: <a href="http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897"><img src="http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897/status.svg"></a>
Markdown: [![status](http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897/status.svg)](http://joss.theoj.org/papers/7af4e58d74943cd41832d1320c83a897)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@melund & @ixjlyons, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @labarba know.

✨ Please try and complete your review in the next two weeks ✨

Review checklist for @melund

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @ixjlyons

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

Review checklist for @alcantarar

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

whedon commented 5 years ago

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @melund, it looks like you're currently assigned as the reviewer for this paper :tada:.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands
whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

labarba commented 5 years ago

👋 @melund, @ixjlyons — Thank you for agreeing to review for JOSS! This is where the action happens: work your way through the review checklist, feel free to ask questions or post comments here, and also open issues in the submission repository as needed. Godspeed!

labarba commented 5 years ago

@ixjlyons — I see that you haven't checked off any items in your review checklist. Have you been able to get a start on this? Let me know if you have any questions!

ixjlyons commented 5 years ago

Sorry I haven't gotten to this yet. I will try to get started this week and complete a review over the weekend.

labarba commented 5 years ago

Hi @ixjlyons — I think you meant to work on this review over the past weekend. Can you give us a status update? We can also set an automatic reminder, if you have a new ETA.

ixjlyons commented 5 years ago

Apologies for the delay. Here is my review:

Paper

The paper is written clearly and offers insight into the functionality of the library. A few minor points:

Docs

The documentation is distributed into the README, a couple other markdown documents, and a Jupyter notebook. These work together to cover much of the library's functionality and usage, but you could benefit from a dedicated documentation site with API documentation generated from the docstrings. Every function appears to have docstrings and those files are browsable through GitHub, so I will defer to @labarba on whether or not that satisfies the functionality documentation check.

A few other minor issues:

Code/Functionality

Unfortunately, I wasn't able to do much with Dynamo being stuck with Linux (no pyrealsense2) and lacking D4XX cameras, so I'm leaving the installation, functionality, and performance checks un-checked.

I notice that some of the modules (e.g. calculate_rmsd.py) can be run as scripts. Consider packaging that functionality as console scripts, or perhaps mention that they can be run this way in the documentation.

Just out of curiosity, have you considered additional storage formats aside from pickle? HDF5 or other formats might work well and could provide better portability.

Other

The tests are not automated, but as far as I know the instructions for running the tests manually suffices for the "automated tests" check. This library poses some challenges for automated testing (with dependence on hardware), though some of the computational aspects of the software could be automatically tested.

Overall

Overall I think Dynamo would benefit from a dedicated documentation page with API documentation and perhaps automated testing of functionality that doesn't depend on presence of specific hardware. The documentation seems fairly complete, but a more cohesive presentation of it could help newcomers to the library.

melund commented 5 years ago

@labarba. I am also done with my review. Specific feedback has been handled on the issue tracker:

Since I haven't got access to the cameras I couldn't review all part of the functionality. But what I could check and review does comply with the requirements of JOSS.

I agree with @ixjlyons about the lack of API documentation, but it is not blocking in my view. I think the most important comment is the lack of a conda-forge package. It would help guarantee that this package continues to work in the future. However, the current state is in accordance with the requirements of JOSS.

abhishektha commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

abhishektha commented 5 years ago

Hi all,

Thank you for the insightful comments and help with getting DynaMo up to JOSS standards so far. I've fixed the errors in the paper and tutorial Jupyter Notebook.

Is there anything else needed from me? I am a little unclear if I need to add to the documentation to proceed with the paper. Would a new section on the github which shows how to call each function of the package meet this requirement? If so I'm happy to get that done soon.

I will also work on getting this package onto conda-forge. It took me a while to catch up on reading how conda-forge recipes work but I think once we have pyrealsense2 on conda-forge, I can easily create a package for DynaMo.

Thanks again for all you contributions so far!

melund commented 5 years ago

It took me a while to catch up on reading how conda-forge recipes work but I think once we have pyrealsense2 on conda-forge, I can easily create a package for DynaMo.

That is great, but I think you will have to initiate the work with a pyrealsense2 conda-forge package. I think a win only package would be Ok for the first iteration. If you start the work, there is a good chance others will chip in and help.

abhishektha commented 5 years ago

@labarba Is there anything specific that I need to do for us to proceed with the review? I believe I addressed all the reviewer's comments (please let me know if I didn't). I am a bit confused from the reviews if I need to improve the documentation to meet the JOSS standards (since @melund mentioned it was non-blocking), so please let me know what is needed from me.

labarba commented 5 years ago

Hi @abhishektha — neither of the two reviewers have been able to check the functionality of the software, due to the hardware dependency (cameras) and lack of access. Accepting the paper under those conditions is awkward enough, so I would request that you make every effort at improvements that you can, given the reviewer comments.

Since you already have docstrings throughout, it should not be too hard to deploy API documentation using Sphinx? Please have a look at that and let me know what you think.

In the meantime, would you be able to suggest someone who may have access to the cameras to provide a check of functionality through a partial & supplementary review?

abhishektha commented 5 years ago

Thanks for the reply @labarba! I can definitely deploy API documentation through Sphinx, and will let you and the reviewers know when that is done.

Would it be okay if we had someone independently review functionality of the code using our own camera setup in our lab? We have a few colleagues in the field who have not worked with us on this project but may be able to provide a check of functionality. We would simply provide them access to our set of cameras and allow them to hook up their own computer to check functionality.

labarba commented 5 years ago

hey @abhishektha ... do you have updates with regards to the documentation? As to your question, if you could get someone to contribute some functionality checks to this review, that would good, give the restrictions we face.

abhishektha commented 5 years ago

@labarba Sorry for the delay, I had some trouble using Sphinx with the numpydoc style of tooltips I had been using, but its all fixed now and we finally have API documentation hosted on Github Pages and linked from the readme. @ixjlyons, @melund, please let me know if this new API documentation meets the standards.

I would like to suggest @alcantarar to contribute functionality checks to the review. He has access to our cameras but has not worked on the development of this project.

labarba commented 5 years ago

👋 @alcantarar — would you be willing to contribute a partial review of this JOSS submission, given that the assigned reviewers both lack access to the needed hardware to confirm functionality?

alcantarar commented 5 years ago

I would be willing to assist with the review. Should I just open issues in the Dynamo repo as needed and comment below when my review is complete? I was going to follow the functionality section of the review checklist you've provided above.

labarba commented 5 years ago

@whedon add @alcantarar as reviewer

whedon commented 5 years ago

OK, @alcantarar is now a reviewer

labarba commented 5 years ago

Hi @alcantarar — thanks for agreeing to provide an additional review. I have added a review checklist for you. If you don't want to check off all items, that is OK, but we do need the COI and CoC responses from you, and especially need the functionality items that the other two reviewers could not check. Godspeed!

labarba commented 5 years ago

👋 @alcantarar — Could we have a status update on your review? Thanks!

alcantarar commented 5 years ago

I've opened a few issues in the target repo that have been/are currently being addressed . I'll continue working on the most recent issue with @abhishektha this week.

alcantarar commented 5 years ago

hey @labarba my review is complete. There are a few open issues in the target repository about documentation, but @abhishektha can close them easily. I went all the way through the Example Tutorial and it functioned as advertised. Really fun using the cameras actually!

I've checked off the COI/COS and Functionality checklist above. Did I miss anything?

abhishektha commented 5 years ago

@labarba, i went through and checked the documentation issues and addressed and closed them. Please let me know if there's anything else you need from me!

labarba commented 5 years ago

@alcantarar — to clarify, you are only contributing a partial review, focusing on the functionality of the software, and do not plan to review the documentation and paper. Correct?

alcantarar commented 5 years ago

@labarba Correct. I only made suggestions about documentation that directly influenced my ability to test the functionality. These issues have been addressed.

labarba commented 5 years ago

@alcantarar — would you mind writing a brief report here about your review of the functionality, with the purpose of informing the other two reviewers, who checked other aspects of the submission? We will ask @melund and @ixjlyons to provide a recommendation of acceptance on the basis of your additional checks of functionality, so I think this would be helpful.

alcantarar commented 5 years ago

The target repository claims to allow users to capture object shape or reflective marker data using multiple Intel RealSense cameras. The hardware requirements and software dependencies in the README.md file are comprehensive and allowed me to use my laptop to test two Intel RealSense cameras. The conda environment also worked as intended in terms of installation.

Setting up the cameras was straightforward and required no additional software to install. Users will be able to plug in the cameras and immediately be able to execute DynaMo functions to calibrate the cameras, stream/save images to their computer, and view 3D images afterwards. During my review, I ran into some issues with the number of parameters required for a given function or instances of hardcoding, but these issues have been addressed. Following the target repository's example page, I was able to record and save 3D object shape data from Intel RealSense Cameras, as the documentation claims.

melund commented 5 years ago

Thanks @alcantarar.

@labarba, I can recommend DynaMo for acceptance. Great work @abhishektha

ixjlyons commented 5 years ago

I also recommend accepting.

abhishektha commented 5 years ago

@labarba Is there anything you need from me to proceed with the paper?

labarba commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

:point_right: Check article proof :page_facing_up: :point_left:

labarba commented 5 years ago

Please see my copy editing of the paper on the linked PR.

labarba commented 5 years ago

@whedon check references

whedon commented 5 years ago 
Attempting to check references...
whedon commented 5 years ago 

OK DOIs

- 10.1007/978-0-387-31439-6_472 is OK
- 10.1107/S0567739476001873 is OK

MISSING DOIs

- https://doi.org/10.1016/j.jbiomech.2008.06.024 may be missing for title: Systematic accuracy and precision analysis of video motion capturing systems–exemplified on the Vicon-460 system

INVALID DOIs

- https://doi.org/10.1016/j.displa.2013.08.011 is INVALID because of 'https://doi.org/' prefix
labarba commented 5 years ago

@abhishekbajpayee Can you now check if the DOIs flagged by whedon need fixing?

abhishektha commented 5 years ago

@whedon check references

whedon commented 5 years ago 
Attempting to check references...
labarba commented 5 years ago

@whedon generate pdf

whedon commented 5 years ago 
Attempting PDF compilation. Reticulating splines etc...
whedon commented 5 years ago

PDF failed to compile for issue #1466 with the following error:

Error reading bibliography ./paper.bib (line 30, column 1): unexpected "u" expecting space, ",", white space or "}" Error running filter pandoc-citeproc: Filter returned error status 1 Looks like we failed to compile the PDF

abhishektha commented 5 years ago

@whedon check references

whedon commented 5 years ago 
Attempting to check references...