[JOSS Review] Paper Feedback

[sea-bass]

Thank you for submitting this cool toolbox! Here is my feedback on the paper.

General Feedback

• I may debate the claim that a Python package should be used for real-time applications. If that was critical, then this package should have been implemented in C++, Rust, or something like that. I think Python is good for ease of use and prototyping, but I may pull back on some of these claims.
• In addition to the citations, which you already have, I think it would be helpful to have a brief introduction to these evaluation metrics (ellipsoids and polytopes) and what they do. As such, I think your "Physical Capacity Metrics" would be better served BEFORE the "Implemented Polytope Evaluation Algorithms" section. First say what these things are, and what they are used for, and then get into which algorithms are in PyCapacity.
• The tables of evaluation times need to have some notion of variance. Instead of saying things like ~5ms, I (and readers) would be mostly interested in something like 5ms +/- 1ms, as well as a brief description of how this data was collected (how many samples, etc.) -- see e.g., https://jhavl.github.io/mmc/ for an example.
• Please add a Conclusion section at the end
• Capitalize Python, Jacobian, and Cartesian throughout

---

Summary

• Typo in line 7: "the robot's are required" --> "robots are often required"
• Typo in line 9: "One of ways to quantify" --> "One way to quantify"
• Line 11: Consider gender-neutral language in line 11 "where he lacks" --> "where they lack"
• Typo in line 13: "as todays collaborative" --> "as today's collaborative"
• Missing : in line 19 -- "Most of these metrics can be represented by two families of geometric shapes: ellipsoids and polytopes"
• Finish your summary by saying what this paper does: It introduces a new package named PyCapacity!

---

Statement of Need

• More of a question: Have there been other tools available, in Python or otherwise, that do these ellipsoid/polytope calculations and visualizations? If so, you should list them and cite them. For example:
  • https://jhavl.github.io/mmc/
  • https://github.com/tecnalia-medical-robotics/manipulability_metrics
  • https://gitlab.com/KeerthiSagarSN/rospygradientpolytope
• Typo in line 27: "human musculoskeletal modes" -> "human musculoskeletal models"
• I really like that you demonstrated which existing publications already use your work. Very important!

---

Physical Capacity Metrics

• Typo in line 83: "Reachable space approximation of in the desired horizon" -- of what? I think it is either missing a word, or the "of" should be deleted
• The caption in Figure 2 is missing a period after "visualized with biorbd".

https://github.com/openjournals/joss-reviews/issues/5670

[askuric]

Hi @sea-bass

Thanks for the detailed review I appreciate it. I helped me a lot to go a step further and I ma pleased with the result.

The paper is almost entirely rewritten, but the main changes are:

• a bit more consequential Statement of need with a discussion about the existing solutions
• a bit more info about what is the main aim of the library (apart form the near real-time)
• I've done a bit more benchmarking and updated the table with the statistics, as well as provided the benchmarking scripts in the examples folder so that everyone can check on their system
• Added the conclusion
• A section about the differences of ellipsoids and polytopes, and what do they actually represent added

In terms of the real-time claim.

It is absolutely true that if the real-time would have been the only goal the python implementation does not make sense. The goal of the library is to provide efficient tools for their evaluation in a easy-to-use package. However, looking at the efficiency of the implemented algorithms , even though they are implemented in python, they are in many cases good enough for interactive applications.

Especially in terms of musculoskeletal models, they are usually restricted to off-line use, due to their inherent dimensionality issues (large number of muscles) . The ICHM algorithm implemented in this package is able however to evaluate their polytopes within a 200-300ms which might have taken hours with different methods. So its much closer to real-time than before, but it is far from sub-millisecond that you'd probably expect form "real-time" claims.

However just as an example, we have used this package (both for robots and humans) for the real-time robot control in our lab, we ran the code at the frequency of 50 to 100Hz which is perfectly enough for many use-cases.

Finally, the area of using task-space physical ability metrics (especially polytopes) in real time applications is very small at the moment, mostly due to their computational complexity, so having the real-time keyword in the title allows to reach further and show to the community that it is not as far from the real-time as they might think. :D

[ShravanTata]

@askuric I have the same concern as @sea-bass on the real-time claim. I do agree with your rationale but may be think about replacing "Real-time" with "Fast" or "Efficient" keywords instead.

[ShravanTata]

• Typo in line 21: "off line" --> "offline"/"off-line"

[sea-bass]

Thanks @askuric -- took a look at your reworked paper and it looks awesome! Much clearer and gentler introduction to the readers.

My only "real" comment is that it's probably risky for the paper to provide direct links to the performance benchmarking scripts since this might change in the future. I'd just say "provided in the repository" and add instructions in the repo itself for reproducing the benchmarking for both the robot and musculoskeletal models.

Finally, some small editorial comments as I read through. Nothing major:

• In the summary, and again in the statement of need, you mention robotic-toolbox but I think they call the module roboticstoolbox. Either use this, or the full name "Robotics Toolbox for Python".
• In the summary, line 15, small typo "biomechanics softwares opensim" --> "biomechanics software opensim"
• Typo in line 57-58: "as they are essentially a triangulated meshes" --> "as they are essentially triangulated meshes"
• Line 65 missing a space between cddlib and the citation, same with line 69 in pygradientpolytope and its citation.
• Typo in like 75: "Futhermore" -> "Furthermore"
• Line 129: Extra space around the comma in "their faces , H-representation".
• Lines 208-209 missing a word: "provides a brief performance evaluation of their execution times"
• Line 212 fix capitalization: "7 dof Franka Emika panda" -> "7 DOF Franka Emika Panda"
• Same with like 218: "50 muscle 7 dof" -> "50 muscle, 7 DOF"
• Lines 213-214 missing a word: "computer equipped with a 1.90 GHz Intel i7..."

[askuric]

Hey @sea-bass,

Thanks a lot for your updates, the paper is updated with your comments. I absolutely agree about the script link, it is not the smartest idea in long-run.

The paper is regenerated in the main issue: https://github.com/openjournals/joss-reviews/issues/5670

Let me know if there is anything else you'd like me to do.

[sea-bass]

LGTM -- thanks for making all these updates!

[askuric]

Hi @ShravanTata, Sorry to rush you, but did you maybe have time to look into the made changes? I'd be happy to hear your feedback. Thanks!