[REVIEW]: Universal Numbers Library: Multi-format Variable Precision Arithmetic Library

[editorialbot]

Submitting author: !--author-handle-->@Ravenwater<!--end-author-handle-- (E. Theodore L. Omtzigt) Repository: https://github.com/stillwater-sc/universal Branch with paper.md (empty if default branch): main Version: 3.68 Editor: !--editor-->@danielskatz<!--end-editor-- Reviewers: @mmoelle1, @mlxd, @yboettcher Archive: 10.5281/zenodo.7735084

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/932fdfc2735a85422e4431f27ebfc0d0"><img src="https://joss.theoj.org/papers/932fdfc2735a85422e4431f27ebfc0d0/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/932fdfc2735a85422e4431f27ebfc0d0/status.svg)](https://joss.theoj.org/papers/932fdfc2735a85422e4431f27ebfc0d0)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@mmoelle1 & @mlxd, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @mmoelle1

📝 Checklist for @mlxd

📝 Checklist for @yboettcher

[danielskatz]

@editorialbot check references

[danielskatz]

btw, these three are all commands you can run as well

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.23919/date.2019.8715262 is OK
- 10.1137/17m1140819 is OK
- 10.1109/mm.2021.3061394 is OK
- 10.1007/978-3-031-09779-9_2 is OK
- 10.1109/sc.2018.00050 is OK
- 10.1145/3148226.3148237 is OK
- 10.1007/978-3-319-93698-7_45 is OK
- 10.1137/18m1229511 is OK
- 10.1007/978-3-031-09779-9_7 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[jamesquinlan]

@danielskatz Thanks! 👍🏼

[jamesquinlan]

@yboettcher does the first paragraph rewrite in the statement of need address your concerns about clarity? Thanks

[yboettcher]

@jamesquinlan I like that paragraph much better now! The only phrasing I'm a bit confused by now is the "high energy consumption requirement" (l. 40). While I think it's totally clear what you want to say here, I have never seen "energy consumption requirement" in a single phrase. Only "energy requirement" or "energy consumption", so I feel like using both at the same time is a bit off.

[jamesquinlan]

@yboettcher thank you for the feedback. I'll rewrite it to change that. I will wait until @mlxd and @mmoelle1 finish their reviews/feedback/concerns to make changes.

[mlxd]

@mlxd @yboettcher @mmoelle1 I have updated the README.md to add extra sections that explain how to use and how to extend the library. This is in the v3.68 branch.

Love to hear your feedback if these additions alleviate your concerns.

Hi @Ravenwater I think this helps address some concerns, though I think there would be benefit from tidying up stray XLSX and equivalent files in docs.

Documentation is provided via `Docs`, including a tutorial...

I think I'd also potentially suggest adding a directory level README.md to docs indicating the contents, and allowing a reader of the paper to easily find the intended tutorial as listed. Lastly, I'd match the case of Docs to that of the directory docs for consistency.

With these, I'm happy to give the all clear.

[Ravenwater]

@mlxd Thank you for that feedback. I have cleaned up the docs directory, added the docs/README.md that explains the content, and added a description of al the new docs directory goodies in the repo README in the section *How to develop and extend Universal".

It is this commit: https://github.com/stillwater-sc/universal/commit/3e957128214920af1fe4a2bf79bb9577f034de31

[Ravenwater]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[mlxd]

@mlxd Thank you for that feedback. I have cleaned up the docs directory, added the docs/README.md that explains the content, and added a description of al the new docs directory goodies in the repo README in the section *How to develop and extend Universal".

It is this commit: stillwater-sc/universal@3e95712

Thanks @Ravenwater I'll resolve the issue https://github.com/stillwater-sc/universal/issues/325 and consider my review checklist complete. Nothing more from my side, and great job on the work in this library.

[Ravenwater]

@danielskatz ship it :-)

[Ravenwater]

Thanks to all reviewers, @mlxd @mmoelle1 @yboettcher for all the hard work and critical feedback: it made the submission better and ready for broader adoption.

[Ravenwater]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[jamesquinlan]

Thanks to all reviewers for valuable feedback (@mlxd @mmoelle1 @yboettcher), and many thanks to @danielskatz .

We really appreciate it and look forward to working with you again in the future. :)

[danielskatz]

@yboettcher and @mmoelle1 - can you let us know where you are on your review, and what else is needed for you to be able to check off your remaining review items?

[yboettcher]

For me, the only thing left to do is (successfully) testing out the different number formats offered by Universal. I wanted to do this like I used the posits back when I wrote my thesis, but most number formats other than posit and fixpnt resulted in compile-time errors (which I then turned into a fix here and a proposed solution here). This might however be because I was using Eigen, while the software paper and universal docs only mention MTL4. So trying out the number formats with MTL4 would be my next (and then also last) step. A last thing I would mention, although that's totally not a blocker: Universal offers SORNs, which is also mentioned by the software paper, but I had a hard time figuring out, what SORNs would be (other than "sets of real numbers"). I eventually found this https://link.springer.com/chapter/10.1007/978-3-031-09779-9_1, but I thought maybe some reference (maybe not even in the paper and just in the Readme) would be great. I would also be fine though if it is deemed out-of-scope and a user is expected to know what they want.

[yboettcher]

I think I would like some guidance here. I uploaded my test cases to https://github.com/yboettcher/universal_tests with most of them being disabled in the CMakeLists.txt. The only one, where I can compile most of the number formats and actually test them is a simple test for +-/. Everything involving Eigen/MTL does not compile with most number formats for me. For SORNs (which are mentioned as supported in the software paper) I cannot compile this simple test, as the compiler complains about missing implementations for +-/. Now, I admit that throwing these types at Eigen/MTL and expecting them to work fine might be a bit too much (after all, especially with Eigen, no one ever claimed there was any compatibility with it. It worked very good for posits though). Now if it were just for the posits: I have used them extensively with Eigen and Bembel and would happily check off any remaining items on the checklist. However, I was trying to gauge all the other formats that universal offers and was surprised they did not work so seamlessly. I think my overarching question here is: Should I check off the functionality (and performance), because universal's main offering, the posits, have worked so well for me, or should I insist on all the other types working too, because the software paper indeed states that universal offers support for them? (once again, not working with Eigen is fine by me, because it never claimed to do so, and someone using universal would just have to do linear algebra manually, but the thing I am most surprised about is most likely the SORNs not working for me at all, although they were explicitely mentioned in the software paper.) As a heads up, the other important number formats (posits, cfloat, logarithmic, and some more) pass this +-*/ test and are fine on my side. There were some other failing tests, but I am not too worried about those.

[danielskatz]

@Ravenwater - can you please offer your comments on the above discussion from @yboettcher ?

[ghost]

@danielskatz @yboettcher I am investigating the Eigen breakage and the PRs Yannick provided. Folks that use Eigen. like GISMO, have typically provided Eigen-specific extensions for their integrations, but I will try to see if we can close this feature if it is isolated to Eigen asking for a specific constructor.

SORNs are provided by a research team from Moritz Bartel from University of Bremen: https://link.springer.com/chapter/10.1007/978-3-031-09779-9_1 and as their paper indicates, SORNs are not to be used as plug-in replacement arithmetic as they exploit very specific behaviors from the encoding, these SORNs are not a general arithmetic.

[ghost]

@yboettcher the compilation error you encounter is correct behavior. The standard library provides functions for the native types, so std::sqrt(someNativeType) will find the sqrt() function. However, if you want to call a specialized function for an extension type, you will need to call the sqrt() function defined for that specialization. For Universal these functions reside in the namespace sw::universal. Thus the pattern that is correct is to define your uses like this:

namespace yourNamespace {
  template<typename Scalar>
  Scalar myKernel() {
    using std::sqrt;
    Scalar a = 1507.2; // arbitrary number
    Scalar b = sqrt(a); // arbitrary function from 
    return a * a;
  }
}

The using std::sqrt is required to match the std namespace functions in case the Scalar is a native type. When the Scalar type is a Universal type, the compiler will be able to infer the namespace it needs to match the argument.

So, your test environment is not using the proper C++ patterns to use Universal.

[Ravenwater]

@yboettcher can you confirm that you agree with my analysis?

[jamesquinlan]

@yboettcher @danielskatz Any thoughts on the latest remarks? Do you agree with @Ravenwater analysis? Thanks

[danielskatz]

It's @yboettcher who needs to comment, not me (at this point)

[Ravenwater]

@yboettcher please respond so that we can move this forward.

[yboettcher]

My apologies for the delay.

I think with @Ravenwater's explanations there are no further concerns on my side. Not working perfectly with Eigen is not an issue, since Universal never claimed to do so. And the explanations for the std:: functions and SORNs also seem reasonable to me.

With that said, I have no further concerns. Well done!

[danielskatz]

Thanks @yboettcher

[danielskatz]

@Ravenwater - ok, now on to the last steps post-review

[danielskatz]

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

• [ ] Double check authors and affiliations (including ORCIDs)
• [ ] Make a release of the software with the latest changes from the review and post the version number here. This is the version that will be used in the JOSS paper.
• [ ] Archive the release on Zenodo/figshare/etc and post the DOI here.
• [ ] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

Editor Tasks Prior to Acceptance

• [ ] Read the text of the paper and offer comments/corrections (as either a list or a PR)
• [ ] Check the references in the paper for corrections (e.g. capitalization)
• [ ] Set article DOI with @editorialbot set <DOI here> as archive
• [ ] Set version with @editorialbot set <version here> as version
• [ ] Double check rendering of paper with @editorialbot generate pdf
• [ ] Specifically check the references with @editorialbot check references and ask author(s) to update as needed
• [ ] Recommend acceptance with @editorialbot recommend-accept

[danielskatz]

@Ravenwater - just to be clear, I'm waiting for you to do the first 4 things above...

[ghost]

@danielskatz :-) thanks for the ping, let's get this done.

[Ravenwater]

Post-Review Checklist for Editor and Authors

Additional Author Tasks After Review is Complete

• [x] Double check authors and affiliations (including ORCIDs)
• [x] Make a release of the software with the latest changes from the review and post the version number here. v3.68 This is the version that will be used in the JOSS paper.
• [x] Archive the release on Zenodo/figshare/etc and post the DOI here. 10.5281/zenodo.7735084
• [x] Make sure that the title and author list (including ORCIDs) in the archive match those in the JOSS paper.

[danielskatz]

@editorialbot set 10.5281/zenodo.7735084 as archive

[editorialbot]

Done! Archive is now 10.5281/zenodo.7735084

[danielskatz]

@editorialbot set 3.68 as version

[editorialbot]

Done! version is now 3.68

[danielskatz]

@editorialbot recommend-accept

This will generate a proof that I will then carefully read and check

[editorialbot]

Attempting dry run of processing paper acceptance...

[editorialbot]

Couldn't check the bibtex because branch name is incorrect: v3.68

[editorialbot]

:warning: Error preparing paper acceptance.

[danielskatz]

@Ravenwater - what happened to the branch (v3.68) that contained the paper source? We don't need that branch, but we do need the paper source

[danielskatz]

It looks like maybe it got merged, and is now in main?

[danielskatz]

@editorialbot set main as branch

[editorialbot]

Done! branch is now main

[danielskatz]

@editorialbot recommend-accept

trying again to generate a proof to check

[editorialbot]

Attempting dry run of processing paper acceptance...

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.23919/date.2019.8715262 is OK
- 10.1137/17m1140819 is OK
- 10.1109/mm.2021.3061394 is OK
- 10.1007/978-3-031-09779-9_2 is OK
- 10.1109/sc.2018.00050 is OK
- 10.1145/3148226.3148237 is OK
- 10.1007/978-3-319-93698-7_45 is OK
- 10.1137/18m1229511 is OK
- 10.1007/978-3-031-09779-9_7 is OK

MISSING DOIs

- None

INVALID DOIs

- None