[PRE REVIEW]: Open-Source Pattern Creation Tool for Medical Extended Reality Image Quality Assessment

[editorialbot]

Submitting author: !--author-handle-->@brendanc490<!--end-author-handle-- (Brendan Collins) Repository: https://github.com/DIDSR/WebXR-tools.git Branch with paper.md (empty if default branch): JOSS-Submission Version: v1.0 Editor: !--editor-->@osorensen<!--end-editor-- Reviewers: @codeling, @osorensen Managing EiC: George K. Thiruvathukal

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/b7c917fb55a1743a539139a74229fdcf"><img src="https://joss.theoj.org/papers/b7c917fb55a1743a539139a74229fdcf/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/b7c917fb55a1743a539139a74229fdcf/status.svg)](https://joss.theoj.org/papers/b7c917fb55a1743a539139a74229fdcf)

Author instructions

Thanks for submitting your paper to JOSS @brendanc490. Currently, there isn't a JOSS editor assigned to your paper.

@brendanc490 if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). You can search the list of people that have already agreed to review and may be suitable for this submission.

Editor instructions

The JOSS submission bot @editorialbot is here to help you find and assign reviewers and start the main review. To find out what @editorialbot can do for you type:

@editorialbot commands

[editorialbot]

Hello human, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1109/VR46266.2020.00108 is OK
- 10.1002/jsid.1208 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[editorialbot]

Software report:

github.com/AlDanial/cloc v 1.88  T=1.85 s (1177.9 files/s, 223210.0 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
JSON                            12              5              0         210852
YAML                             7              1              9          97436
JavaScript                      47           8132           5941          41744
CSS                             28           5967            238          16478
SVG                           2023              0             15           8089
Sass                            19            556             72           6965
LESS                            18            552             72           6907
HTML                             5            106             51           1103
reStructuredText                 6             75             18            347
Markdown                         5             50              0            248
DOS Batch                        1              8              1             26
TeX                              1              1              0             16
Python                           1             14             31             10
TypeScript                       1              4              0             10
make                             1              4              7              9
-------------------------------------------------------------------------------
SUM:                          2175          15475           6455         390240
-------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

[editorialbot]

Wordcount for paper.md is 871

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[Kevin-Mattheus-Moerman]

@brendanc490 thanks for this submission. I am the AEiC for this track and here to help process the initial steps. For the moment I've flagged this work for a scope review by our editorial board. In particular I need some help to see if this work conforms to what we class as research software. The paper mentions that patterns created through the website could serve to help "approve devices for medical use". This could be an important application. However this functionality does not make it research software. Could you help provide some comments here as to how you think your software would be used in scientific research? It would help if what you describe paints a clear picture as to why this work is important to scientific research, i.e. such that one would feel the need to cite it in a subsequent paper. Utility tools are typically not cited, and do not require a paper. However software that has a direct impact on scientific findings would definitely need to be cited. As you address this query, you may also decide to amend your paper and statement of need section on the research applications of this work. Thanks.

[brendanc490]

Hi Kevin,

This submission is coming on behalf of the FDA, so the tool is geared towards regulatory science. We have developed a regulatory science tool that enables the measurements of head-mounted display (HMD) quality by creating customized scenes and patterns. This is in line with the definition of regulatory science as it would fall under the creation of tools for the assessment of the quality of FDA-related devices. This tool in particular provides an easy way to display scenes on multiple headsets so that we can assess and compare image quality. This is accomplished through the use of known standardized patterns like the grille or dot array and simple entities like circles, rectangles, or triangles. This flexibility allows for the conduction of targeted examinations of specific display characteristics. For instance, we can create and share packages that are designed to assess display resolution, chromatic aberrations, color mapping, and other spatiotemporal effects. This tool also addresses the issue of pattern portability. Currently, HMDs are very heterogeneous, making it difficult to deploy the same pattern to multiple headsets. This barrier often comes up in standards meetings as an impeding force on the development and validation of different HMDs. In short, this tool provides scientists, medical device manufacturers, and regulators access to a centralized scene to test various display devices and provide reports that are easier to understand and, most importantly, replicate. Our goal was to have a referenceable tool that we could use to guide manufacturers or scientists when asking for specific image quality measurements on new VR/AR devices.

Also, the paper has been updated to include this information. Let me know if you have any further comments or questions.

Thanks, Brendan

On Mon, Sep 4, 2023 at 11:55 AM Kevin Mattheus Moerman < @.***> wrote:

@brendanc490 https://github.com/brendanc490 thanks for this submission. I am the AEiC for this track and here to help process the initial steps. For the moment I've flagged this work for a scope review by our editorial board. In particular I need some help to see if this work conforms to what we class as research software. The paper mentions that patterns created through the website could serve to help "approve devices for medical use". This could be an important application. However this functionality does not make it research software. Could you help provide some comments here as to how you think your software would be used in scientific research? It would help if what you describe paints a clear picture as to why this work is important to scientific research, i.e. such that one would feel the need to cite it in a subsequent paper. Utility tools are typically not cited, and do not require a paper. However software that has a direct impact on scientific findings would definitely need to be cited. As you address this query, you may also decide to amend your paper and statement of need section on the research applications of this work. Thanks.

— Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/5798#issuecomment-1705478952, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQJNVRANTQ4PBVEPKTWPTJ3XYX2WTANCNFSM6AAAAAA4IJ6KIU . You are receiving this because you were mentioned.Message ID: @.***>

[arfon]

@brendanc490 – could you also explain how this repository is structured? I'm seeing a lot of JavaScript which I think is the bulk of the software contribution here?

On quick inspection, it's not clear how to test the software. Could you explain how that is accomplished here?

[brendanc490]

I’d be happy to describe the repository. We submitted under the JOSS-Submission branch. The tool we are submitting is the Custom Pattern Creation Tool located in the "Custom" folder. It draws functionality from the Compatibility tool to interface with virtual reality controllers and headsets, which is also included in the "Compatibility" folder. The majority of work was done in JavaScript as you pointed out, though there is some HTML and CSS work. This encompasses everything except for the contents of the "lib" folder which contains code libraries used in the project.

Upon opening the "Custom" folder you will see a readme containing detailed steps on how to test the tool. We opted for manual tests since the tool is focused on providing a user interface. The tests given highlight all the core functionalities of the tool including creating, editing, saving, and sharing a pattern.

Let me know if you have any questions. Brendan

On Mon, Oct 2, 2023 at 5:03 AM Arfon Smith @.***> wrote:

@brendanc490 https://github.com/brendanc490 – could you also explain how this repository is structured? I'm seeing a lot of JavaScript which I think is the bulk of the software contribution here?

On quick inspection, it's not clear how to test the software. Could you explain how that is accomplished here?

— Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/5798#issuecomment-1742650304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQJNVRAZZDCFWYBMA6FMYFDX5J7MNANCNFSM6AAAAAA4IJ6KIU . You are receiving this because you were mentioned.Message ID: @.***>

[Kevin-Mattheus-Moerman]

@brendanc490 I am pleased to inform you that this work has passed our initial scope review. We will now proceed to find a handling editor.

[Kevin-Mattheus-Moerman]

@editorialbot invite @osorensen as editor

[editorialbot]

Invitation to edit this submission sent!

[osorensen]

@editorialbot assign @osorensen as editor

[editorialbot]

Assigned! @osorensen is now the editor

[osorensen]

👋 @codeling, @ionlights, @jackbrookes, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[codeling]

Yes I can review the submission!

[osorensen]

@editorialbot add @codeling as reviewer

[editorialbot]

@codeling added to the reviewers list!

[codeling]

First quick remark: It seems that the used license (CC0-1.0) is not OSI-approved.

[osorensen]

Thanks a lot for pointing this out @codeling. I'll check this further with the editorial team and see how we proceed.

[osorensen]

@brendanc490, as pointed out by @codeling, the license CC0-1.0 is not OSI approved. Since this is a requirement for JOSS submissions, we cannot review this submission unless you change to an OSI approved license.

Please let us know in this thread whether you change the license. If so, we can proceed with the review.

[osorensen]

@brendanc490, have you decided whether you will change the license or withdraw the publication?

[brendanc490]

We have decided to change the license to The Unlicense. This has been updated in the repository.

On Mon, Oct 16, 2023 at 9:44 AM Øystein Sørensen @.***> wrote:

@brendanc490 https://github.com/brendanc490, have you decided whether you will change the license or withdraw the publication?

— Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/5798#issuecomment-1764517172, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQJNVRHLYGD2ASOW35RPXQ3X7U25PAVCNFSM6AAAAAA4IJ6KIWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRUGUYTOMJXGI . You are receiving this because you were mentioned.Message ID: @.***>

[osorensen]

Thanks @brendanc490, then I'll continue looking for another reviewer.

[osorensen]

👋 @kyq, @sandeepchowdary7, would any of you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[osorensen]

@editorialbot generate pdf

[editorialbot]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[editorialbot]

Five most similar historical JOSS papers:

Virtual Scanner: MRI on a Browser Submitting author: @imr-framework Handling editor: @arokem (Retired) Reviewers: @nstikov, @vsoch, @mathieuboudreau Similarity score: 0.8189

Open Source Optical Coherence Tomography Software Submitting author: @spectralcode Handling editor: @arfon (Active) Reviewers: @jdavidli, @brandondube Similarity score: 0.8142

open_iA: A tool for processing and visual analysis of industrial computed tomography datasets Submitting author: @codeling Handling editor: @katyhuff (Retired) Reviewers: @trallard, @behollister Similarity score: 0.8121

Reel1.0 - A visualization tool for evaluating powder diffraction refinements Submitting author: @fgjorup Handling editor: @rkurchin (Active) Reviewers: @cmbiwer, @mikapfl Similarity score: 0.8106

DetectorChecker: analyzing patterns of defects in detector screens Submitting author: @tomaslaz Handling editor: @fboehm (Active) Reviewers: @janfreyberg, @craddm Similarity score: 0.8072

⚠️ Note to editors: If these papers look like they might be a good match, click through to the review issue for that paper and invite one or more of the authors before before considering asking the reviewers of these papers to review again for JOSS.

[osorensen]

👋 @imr-framework, would you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[osorensen]

👋 @tomaslaz, would you be willing to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html

[osorensen]

@brendanc490, I have a hard time finding a second reviewer for this submission. Do you have any suggestions for potential reviewers?

[brendanc490]

Unfortunately, I don't have any suggestions for you. What is the best way to proceed?

On Fri, Oct 27, 2023 at 4:52 AM Øystein Sørensen @.***> wrote:

@brendanc490 https://github.com/brendanc490, I have a hard time finding a second reviewer for this submission. Do you have any suggestions for potential reviewers?

— Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/5798#issuecomment-1782544393, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQJNVRAFTX73WAY7E3YOWRTYBNY4BAVCNFSM6AAAAAA4IJ6KIWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBSGU2DIMZZGM . You are receiving this because you were mentioned.Message ID: @.***>

[osorensen]

@brendanc490, I'll add myself as a reviewer, so that we can proceed.

[osorensen]

@editorialbot add @osorensen as reviewer

[editorialbot]

@osorensen added to the reviewers list!

[osorensen]

@editorialbot start review

[editorialbot]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/6021.