JOSS paper content - Githubissues

This issue is part of my JOSS review, https://github.com/openjournals/joss-reviews/issues/5756

When I read your paper, the main on-its-face takeaway is that using the native or as-provided interface to Zemax leaves room for improvement, and the principal contribution of ZOSPy is a better/easier to use one.

For example, quoting

With ZOSPy, we aim to provide an accessible interface to the OpticStudio API, enabling the user to focus on optical modelling instead of complex coding. Furthermore, we hope that the easy integration of the ray tracing workflow from optical design to analyses into Python scripts or Jupyter Notebooks facilitates open science.

By placing about a page of Python code as the first "what is it and how to use it" for the paper, that particular example in its whole is given quite a bit of significance. In that example, the lens data manager is filled in programmatically, with

input_beam = oss.LDE.InsertNewSurfaceAt(1)
input_beam.Thickness = 10
# Make a 10 mm thick lens with a radius of curvature of 30mm
# and material type BK10
front_surface = oss.LDE.GetSurfaceAt(2)
front_surface.Radius = 30
front_surface.Thickness = 10
front_surface.SemiDiameter = 15
front_surface.Material = "BK10"
back_surface = oss.LDE.InsertNewSurfaceAt(3)
back_surface.Radius = -30

This would be equivalent to, for example with Code V MacroPlus

INS S1
THI S2 10
INS S2
RDY S2 30
THI S2 10
GLA S2 BK10
INS S3
RDY S3 -30

So, a takeaway that one is inclined to make is that the prescription is stored in python form. The paper also criticizes the .zmx file format as proprietary, which re-inforces this idea. It is not clear that there is an alternative offered by ZOSPy to this; a python script creating a prescription in a non-uniform way is not really a "file format." So while the ZOSPy non-alternative is open source and cannot be proprietary by its nature, I don't think a solution to this problem has been presented, don't think it should be in the paper if a remedy to the problem is not present.

The paper also talks about "small number of" optical systems for the world without ZOSPy, which I find to be a strange point in two ways:

1) non-survey papers would concern themselves only with a small number of systems anyway (the ones being studied)

2) an irregular form of prescription entry would seem in conflict with analyzing large numbers of systems in an automated way

Claims are also made to ZOSPy being pythonic, but it seems most or all of the API is in CamelCase, which python only uses for class names, other variables being in snake_case. So, for example, oss.LDE.insert_new_surface_at(3), instead of the above.

Figure 2 is also not taken from the cited papers (I looked, having a copyright related concern about that), and is probably not generated easily using only ZOSPy? Neither the paper nor the docs (README, related #35 ) show how to extract ray path data with ZOSPy. I think the documentation should either cover how to generate similar graphics, or the paper be made more explicit about what portion of Fig 2 comes from ZOSPy, and what portion is made with other software.

Please do not misunderstand, I think something better than what Ansys provides directly is worth creating and worth publishing. But I think that the software should stand on its own merits, and the criticisms made of alternatives in the paper should be directly addressed (remedied) by ZOSPy

Hi @brandondube, first of all, thank you for your elaborate review! We will work on the feedback and get back to you as soon as possible!

So, a takeaway that one is inclined to make is that the prescription is stored in python form. The paper also criticizes the .zmx file format as proprietary, which re-inforces this idea. It is not clear that there is an alternative offered by ZOSPy to this; a python script creating a prescription in a non-uniform way is not really a "file format." So while the ZOSPy non-alternative is open source and cannot be proprietary by its nature, I don't think a solution to this problem has been presented, don't think it should be in the paper if a remedy to the problem is not present.

The aim of ZOSPy is not provide a new file format, but rather to provide a way of documenting the entire process from creating an optical system, defining the settings of analyses and performing these analyses. This way, the entire process can be shared in an insightful manner. We have altered the last paragraph of the statement of need to clarify this:

With ZOSPy, we aim to provide an accessible interface to the OpticStudio API, enabling the user to focus on optical modelling instead of complex coding. Furthermore, as users can directly share their Python scripts or Jupyter Notebooks rather than screenshots or optical systems saved in proprietary file formats, we strive to facilitate open science.

See also e96e5e48416ded57e8fc136b87d0950c4b74b638.

The paper also talks about "small number of" optical systems for the world without ZOSPy, which I find to be a strange point in two ways:

non-survey papers would concern themselves only with a small number of systems anyway (the ones being studied)

an irregular form of prescription entry would seem in conflict with analyzing large numbers of systems in an automated way

In some applications of ray tracing, e.g. designing a camera lens system, indeed only a limited set of systems is evaluated. Ray tracing is, however, also used in other contexts such as medical research. In that context, it is used to for example evaluate optical properties of the human eye[ref] or to evaluate the interplay between ocular anatomy and an artificial lens [ref]. Where the clinical studies in this area often evaluate over 1000 different eyes, their ray-tracing counter parts are limited to only a small number of patients. We have clarified this in the text:

Although OpticStudio offers an API, the ZOS-API, using this API in Python is complex and time-consuming. It involves, for example, establishing a connection with the API through the .NET framework, casting between .NET and Python datatypes, identifying which constants need to be set in specific cases, and working around non-uniform methods of parsing the output [@GettingStartedWithZOSAPI]. This leads to studies which, in practice, largely rely on user interaction and therefore mostly assess a small number of optical systems. Although this is generally sufficient for the design of an optical system, other applications would benefit from the evaluation a large set of systems. In vision science, for example, clinical studies typically analyze vision-related complaints in cohorts of hundreds of eyes [@Ellis2001; @Osher2008], but the ray-tracing studies aiming to link these complaints to the subject’s ocular optics are limited to a small number of eyes [Holladay1999; @Simpson2020]. Furthermore, the sharing of methodology is often limited to screenshots [@Hong2011], simple tables [@Naeem2022] or optical systems saved in the proprietary file format of OpticStudio [@Polans2018], which hinders open science initiatives.

See also af1195c97b660dc05e3e9a7da5cf2f1d27ca0b9b.

Claims are also made to ZOSPy being pythonic, but it seems most or all of the API is in CamelCase, which python only uses for class names, other variables being in snake_case. So, for example, oss.LDE.insert_new_surface_at(3), instead of the above.

We agree that a part of ZOSPy deviates from PEP8. This is a conscious choice, as we aim to keep the naming in line with the naming within the ZOS-API. We see that this makes the naming within ZOSPy not fully Pythonic. We have removed that statements from the README:

Wrapper around the Ansys Zemax OpticStudio API that provides an intuitive way to interact with the ZOS-API through python using a .NET connection. It thereby allows you to do more optics modelling with less coding.

See also f34842b46195ce0dc0276d7c89424638e742bf09.

Figure 2 is also not taken from the cited papers (I looked, having a copyright related concern about that), and is probably not generated easily using only ZOSPy? Neither the paper nor the docs (README, related #35 ) show how to extract ray path data with ZOSPy. I think the documentation should either cover how to generate similar graphics, or the paper be made more explicit about what portion of Fig 2 comes from ZOSPy, and what portion is made with other software.

Figure 2 is indeed not taken directly from these papers but is a different rendering of the same results (due to copyright concern). For figure 2A, we have indeed postprocessed the image slightly, as we have merged the results of our analysis script (as provided in this example) with a rendering of the eye model in other software. We agree that that is not direct ZOSPy output. Figure 2B, however, is a rendering taken from the non-sequential Shaded Model analysis in OpticStudio made through ZOSPy. We have dropped figure 2A and adjusted the text under Use case and within figure caption 2 to better reflect this:

Furthermore, ZOSPy has been used in different ophthalmic studies. In one of these studies, ZOSPy was used to evaluate the relation of ocular anatomy to peripheral visual complaints [@vanVught2022]. In another study, ZOSPy showed that the extent of an intra-ocular tumor can be overestimated during surgery due to its shadow (\autoref{fig:2}) [@Jaarsma2023].

_Simulation mimicking the clip surgery for radiotherapy of an intraocular tumor [@Jaarsma2023]. The ocular geometry including the dimension of the tumor were loaded into OpticStudio using ZOSPy and the CAD Part: STL object type, after which the retinal illumination was simulated. The results were rendered using the non-sequential Shaded Model analysis (zospy.analyses.systemviewers.nsc_shaded_model)._

See also 6bce3dbda5f29ddcef937d2eea316014129861f1 and f4b11b88f6e562fe723945fc2075f63400b3d62e.

Please do not misunderstand, I think something better than what Ansys provides directly is worth creating and worth publishing. But I think that the software should stand on its own merits, and the criticisms made of alternatives in the paper should be directly addressed (remedied) by ZOSPy

Thank you for your careful evaluation which helps us to make it a better manuscript.

The aim of ZOSPy is not provide a new file format, but rather to provide a way of documenting the entire process from creating an optical system, defining the settings of analyses and performing these analyses. This way, the entire process can be shared in an insightful manner. We have altered the last paragraph of the statement of need to clarify this:

With ZOSPy, we aim to provide an accessible interface to the OpticStudio API, enabling the user to focus on optical modelling instead of complex coding. Furthermore, as users can directly share their Python scripts or Jupyter Notebooks rather than screenshots or optical systems saved in proprietary file formats, we strive to facilitate open science.

I guess it is not clear to me that this is something "new" with this package. If there were no python API at all, one could share a .ZMX file, and .ZPL macros for analysis. One may feel (including myself) that Python > ZPL, and so there is a benefit. And especially that ZOSPy > ZOS-API. But I don't think the "sharing" part is new, moreso that scripts, etc, shared are less noisy / easier to read and comprehend. The closing paragraph, which criticizes sharing alternatives in proprietary formats I still feel is not a real difference; so long as you are wed to OpticsStudio (proprietary), then the script cannot be used without proprietary software anyway, and whether the optical design was in a proprietary .ZMX file or otherwise is rather moot.

Perhaps, as an example, this could be edited to emphasize legibility

With ZOSPy, we provide an easy to use and accessible interface to the OpticStudio API, enabling the user to focus on optical modelling instead of complex coding. Colleagues who are not familiar with the intricacies of the ZOS-API interface are able to read and comprehend scripts that use ZOSPy, providing greater accessibility to analysis conducting with ZOSPy, compared to directly using ZOS-API.

Just a thought

It may help de-emphasize this (which i think is not your central point) to just show fig 1, and leave the minimum working example code to an example file, which can be referenced in the paper's text

...Where the clinical studies in this area often evaluate over 1000 different eyes, their ray-tracing counter parts are limited to only a small number of patients. We have clarified this in the text:

I guess I do not really see, or agree with the alternatives being contrasted here. Zemax can perform Monte Carlo analysis, in which many (100s) of perturbations of an optical system can be generated and analyzed in an automated way. When the systems can be described in the same set of surfaces, but different parameters, the use case you describe seems to map quite neatly to the MC functionality in Zemax, modulo that you do not desire randomness. To me, it seems like a strange angle to contrast your software by implying that Zemax can't do analysis of large ranges of optical systems. This is a fairly core part of designing something through all the way to fabrication, where MC or something like it is done to derive and verify the tolerances.

I am well satisfied with the other changes

Thanks!

Hi @brandondube, thank you for the swift reply. We are happy to hear that you are satisfied with most of our changes! We have made some additional changes following your latest feedback:

I guess it is not clear to me that this is something "new" with this package. If there were no python API at all, one could share a .ZMX file, and .ZPL macros for analysis. One may feel (including myself) that Python > ZPL, and so there is a benefit. And especially that ZOSPy > ZOS-API. But I don't think the "sharing" part is new, moreso that scripts, etc, shared are less noisy / easier to read and comprehend. The closing paragraph, which criticizes sharing alternatives in proprietary formats I still feel is not a real difference; so long as you are wed to OpticsStudio (proprietary), then the script cannot be used without proprietary software anyway, and whether the optical design was in a proprietary .ZMX file or otherwise is rather moot.

Perhaps, as an example, this could be edited to emphasize legibility

With ZOSPy, we provide an easy to use and accessible interface to the OpticStudio API, enabling the user to focus on optical modelling instead of complex coding. Colleagues who are not familiar with the intricacies of the ZOS-API interface are able to read and comprehend scripts that use ZOSPy, providing greater accessibility to analysis conducting with ZOSPy, compared to directly using ZOS-API.

Just a thought

We have changed this section as proposed:

With ZOSPy, we provide an easy-to-use and accessible interface to the OpticStudio API, enabling the user to focus on optical modelling instead of complex coding. As a result, those who are not familiar with the intricacies of the ZOS-API interface will be able to read and comprehend scripts that use ZOSPy. Thereby, ZOSPy provides greater accessibility to conducting analyses in OpticStudio through Python than directly using the ZOS-API.

It may help de-emphasize this (which i think is not your central point) to just show fig 1, and leave the minimum working example code to an example file, which can be referenced in the paper's text

We agree that the entire example code to much for this paper. However, we feel that showing some of the code shows the ease-of-use and comprehensibility of the resulting code. We have therefore shorted the example code to only show some essential lines of code and have referred to the example script for the remainder of the code:

_These examples provide new users with an easy start with ZOSPy. Part of a simple example of using ZOSPy to create and evaluate a thick lens is shown below, and the corresponding results are shown in \autoref{fig:1}._

#...

# Make a 10 mm thick lens with a radius of curvature of 30 mm 
# and material type BK10 
front_surface = oss.LDE.GetSurfaceAt(2)
front_surface.Radius = 30
front_surface.Thickness = 10
front_surface.SemiDiameter = 15
front_surface.Material = "BK10"

back_surface = oss.LDE.InsertNewSurfaceAt(3)
back_surface.Radius = -30
back_surface.Thickness = 29
back_surface.SemiDiameter = 15

#...

# Render the model
draw3d = zp.analyses.systemviewers.viewer_3d(oss)

# Analyze the system by calculating the Modulation Transfer Function (MTF) 
# and the Point Spread Function (PSF)
mtf = zp.analyses.mtf.fft_through_focus_mtf(oss, sampling="512x512", deltafocus=2.5, numberofsteps=51)

huygens_psf = zp.analyses.psf.huygens_psf(oss, pupil_sampling="512x512", image_sampling="512x512", normalize=True)

I guess I do not really see, or agree with the alternatives being contrasted here. Zemax can perform Monte Carlo analysis, in which many (100s) of perturbations of an optical system can be generated and analyzed in an automated way. When the systems can be described in the same set of surfaces, but different parameters, the use case you describe seems to map quite neatly to the MC functionality in Zemax, modulo that you do not desire randomness. To me, it seems like a strange angle to contrast your software by implying that Zemax can't do analysis of large ranges of optical systems. This is a fairly core part of designing something through all the way to fabrication, where MC or something like it is done to derive and verify the tolerances.

we understand and agree with you point. We have therefore rephrased these sentences to better explain the difference between Monte Carlo type of analyses which are possible, and the non-random permutations required for some medically oriented studies:

Although OpticStudio offers an API, the ZOS-API, using this API in Python is complex and time-consuming. It involves, for example, establishing a connection with the API through the .NET framework, casting between .NET and Python datatypes, identifying which constants need to be set in specific cases, and working around non-uniform methods of parsing the output [@GettingStartedWithZOSAPI]. This leads to studies which, in practice, largely rely on user interaction. Although OpticStudio can perform Monte Carlo analyses, where a large number of random perturbations of the system are generated and analysed in an automated way, this type of automation is not suitable when large sets of specific, non-random, combinations of parameters need to be analysed. In vision science, for example, ray tracing is used to design artificial lenses for the eye [@Ellis2001; @Holladay1999], but their evaluation in a large set of patients is hindered as the anatomical parameters of each subject’s eye need to be entered manually. As a result, clinical studies typically describe vision-related complaints in cohorts of hundreds of eyes [@Alfonso2007], but the ray tracing studies aiming to link these outcomes to the subject’s ocular optics are limited to a small number of eyes [@Simpson2020; @vanVught2020].

Apologies, looks like I missed your reply. I am satisfied with the changes, thank you for your work and willingness to modify the paper!

Thank you for your review!

MREYE-LUMC / ZOSPy

JOSS paper content #36