[PRE REVIEW]: Scripts for Easier Use of Spice (SEUS): A Perl script package for simulating and creating batches of circuit netlists for Monte Carlo simulations when using Ngspice or Ngspice-based simulators

[whedon]

Submitting author: @miketuri (Michael Turi) Repository: https://bitbucket.org/miketuri/perl-spice-sim-seus/ Version: v1.0.0 Editor: @VivianePons Reviewers: @mzszym, @victorvalgenti

Author instructions

Thanks for submitting your paper to JOSS @miketuri. Currently, there isn't an JOSS editor assigned to your paper.

@miketuri if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Attempting to check references...

[whedon]

Attempting PDF compilation. Reticulating splines etc...

[whedon]

OK DOIs

- 10.1016/j.sse.2003.12.030 is OK
- 10.1109/CCWC.2017.7868469 is OK
- 10.1109/TCSII.2019.2922921 is OK
- 10.1016/j.vlsi.2016.12.003 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[kthyng]

Hi @miketuri! I have a few questions to learn more about your submission.

• what specific workflows are enabled by your tool? I'd like to know more about why an additional tool on top of the base software being used is needed. Is there a large audience for this tool?
• why can't it be merged into SPICE itself?
• how does it support provenance and model intercomparison or verification and validation studies?

[danielskatz]

👋 @miketuri - note that we are waiting for your responses

[kthyng]

Hi @miketuri — will you have a chance to respond to the questions above? If you won't have time in the next week or two, I propose we pause this submission for now, to be picked back up later when you have time.

[miketuri]

Hello @kthyng, I'm sorry for my late reply; I have been out of town for a few days. Here are my answers to your questions, please let me know if you have any additional questions.

• what specific workflows are enabled by your tool? I'd like to know more about why an additional tool on top of the base software being used is needed. Is there a large audience for this tool? This package under consideration enables statistical or Monte Carlo circuit analysis, measurement support, and alternative output formats for versions of spice which do not support these. Specifically, I was using an older, non-open source variant of ngspice which did not have this support; this version of ngspice was compiled with a model to support simulating FinFET/tri-gate transistors. Models are often made to be used independently of the spice simulator, but as more non-traditional transistor or nanotechnology models are developed, these can necessitate spice simulator modifications (especially if the device does not fit into the four terminals (Drain, Gate, Source, Body) of a MOSFET or terminals of other Spice-supported devices (e.g. BJTs, resistors, capacitors, etc.)) and affect more and more users. Without this package, I would not have been able to run Monte Carlo simulations. I would have been able to make circuit measurements, but this would have been with a waveform viewer such as Mentor Graphics EZWave.

• why can't it be merged into SPICE itself? More recent releases of ngspice have added support for statistical or Monte Carlo circuit analysis by supporting randomized parameters; this is done differently by instead using additional definitions in the spice netlist to give parameters different random values each time the netlist is simulated. More recent releases of ngspice have also added support for measurements by supporting .measure statements similar to other versions of spice. One benefit of this package under consideration is that it can be run on top of ngspice or other spice simulators to perform Monte Carlo simulations and measurements without committing to a specific spice syntax for these behaviors (since ngspice commands can be different from hspice, pspice, etc.).

• how does it support provenance and model intercomparison or verification and validation studies? I'm not sure what you mean by provenance or intercomparison, but the Monte Carlo simulation support would allow circuit designers the ability to verify and validate the correct operation of their circuit or system when impacted by parameter, voltage, and/or temperature (PVT) variations as well as measure/calculate the range of the circuit or system's performance (speed/delay, power/energy, leakage, etc.).

[kthyng]

/ooo today until January 23, 2019

[ooo[bot]]

:+1: Marked @kthyng as OOO from Wednesday, January 15th 2020 to Wednesday, January 16th 2019. :calendar:

[kthyng]

/ooo until January 23, 2019

[kthyng]

/ooo January 14 until January 23

[ooo[bot]]

:+1: Marked @kthyng as OOO from Tuesday, January 14th 2020 to Thursday, January 23rd 2020. :calendar:

[kthyng]

There we go. @miketuri I'll get back to you next week after several big work things and the start of the semester. Thanks!

[kthyng]

@arfon It's weird that other reviews are being tied in here because of ooo references I think

[kthyng]

@miketuri Sorry for the delay. I'd like to follow up on your responses, keeping in mind that I am not in your field of expertise. Are you saying that this submission address a deficiency in former versions of Spice, but is not as relevant for current versions?

I'm discussing with the editorial board to help determine how well this fits in scope. Can you tell me about how a user of your software (and therefore also a reviewer) would test to make sure the code is working correctly? We're also concerned about how easy the package would be to use in practice, and how realistic it is that another person could contribute/change the code, that is, how extensible the package is. You can see more about submission requirements here: https://joss.readthedocs.io/en/latest/submitting.html

[arfon]

@arfon It's weird that other reviews are being tied in here because of ooo references I think

@kthyng the OoO bot is managed at the repository level (openjournals/joss-reviews) so if you've marked yourself as out of the office on any of the review issues then the bot will respond on all of them. Does that make sense?

[miketuri]

Hello @kthyng, thank you for your questions. It is not a problem with the delay; my semester has just started as well. I have provided answers/explanations to your questions below:

@kthyng Are you saying that this submission address a deficiency in former versions of Spice, but is not as relevant for current versions?

Yes, this is correct. Without this Perl package, I would not have been able to perform Monte Carlo circuit simulations with the version of ngspice I was using (this was a closed-source version tied to a particular FinFET transistor model that I needed to use). This Perl package can similarly enable other users of older versions of ngspice to perform Monte Carlo simulations to analyze the effects of parameter variations on circuits.

Current versions of ngspice do support Monte Carlo simulations, however, this is handled differently compared to this Perl package. Current versions of ngspice require a user define his/her random parameters in the circuit netlist file, and each time a simulation is run the parameters will have different randomized values. This Perl package creates a batch of circuit netlist files with randomized values for any defined random parameters--the parameter values for each file are constant, however, you can obtain similar behavior when simulating the batch of randomized circuit netlist files (with this Perl package) versus repeatedly simulating one circuit netlist file with randomized parameters which change for each simulation (as current versions of ngspice do).

Lastly, this Perl package will still work with current versions of ngspice (a user could still use this Perl package to run Monte Carlo simulations instead of the native support in newer ngspice). However, it duplicates the type of Monte Carlo simulation functionality provided by current versions of ngspice.

@kthyng Can you tell me about how a user of your software (and therefore also a reviewer) would test to make sure the code is working correctly? We're also concerned about how easy the package would be to use in practice, and how realistic it is that another person could contribute/change the code, that is, how extensible the package is.

To test this software, a user should use a version of ngspice (the current version will work fine) and simulate a test circuit netlist file--this should behave identically (generate identical output) compared to normal ngspice usage. To test the Monte Carlo simulation support, parameters should be declared to be randomized (in the vary definitions file), then init_batch_spice.pl should be run to create the randomized circuit netlist files. The 0.i file is functionally equivalent to the original circuit netlist file; if this is simulated, it should behave identically (generate identical output) compared to the original circuit netlist file. Other circuit netlist files (with different numbers, e.g. 1.i, 2.i, ...) will have different randomized parameters, but performing a diff with a randomized circuit netlist file with the 0.i file should only show differences with the randomized parameters. Performing these actions can test the code to ensure circuit netlist files are being correctly generated.

[kthyng]

@kthyng Are you saying that this submission address a deficiency in former versions of Spice, but is not as relevant for current versions?

Yes, this is correct. Without this Perl package, I would not have been able to perform Monte Carlo circuit simulations with the version of ngspice I was using (this was a closed-source version tied to a particular FinFET transistor model that I needed to use). This Perl package can similarly enable other users of older versions of ngspice to perform Monte Carlo simulations to analyze the effects of parameter variations on circuits.

I'm concerned about this submission meeting the submission requirement of "The software should be a significant contribution to the available open source software that either enables some new research challenges to be addressed or makes addressing research challenges significantly better (e.g., faster, easier, simpler)." (https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements) if the problem your code addresses is not present in newer version of ngspice, which I suppose is what people are typically using, or at least have the ability to use if they want this functionality.

@kthyng Can you tell me about how a user of your software (and therefore also a reviewer) would test to make sure the code is working correctly? We're also concerned about how easy the package would be to use in practice, and how realistic it is that another person could contribute/change the code, that is, how extensible the package is.

To test this software, a user should use a version of ngspice (the current version will work fine) and simulate a test circuit netlist file--this should behave identically (generate identical output) compared to normal ngspice usage. To test the Monte Carlo simulation support, parameters should be declared to be randomized (in the vary definitions file), then _init_batchspice.pl should be run to create the randomized circuit netlist files. The 0.i file is functionally equivalent to the original circuit netlist file; if this is simulated, it should behave identically (generate identical output) compared to the original circuit netlist file. Other circuit netlist files (with different numbers, e.g. 1.i, 2.i, ...) will have different randomized parameters, but performing a diff with a randomized circuit netlist file with the 0.i file should only show differences with the randomized parameters. Performing these actions can test the code to ensure circuit netlist files are being correctly generated.

Another requirement for submission to JOSS is "your software should be full-featured, well-documented, and contain procedures (such as automated tests) for checking correctness." (https://joss.readthedocs.io/en/latest/submitting.html#typical-paper-submission-flow) so there would need to be a clear description of the testing procedure explained in documentation for a user to be able to follow to make sure their installation of your package is working properly.

I haven't done an exhaustive check of your submission to see if there might be other sticking points, but the first would mark this submission of out of scope if my reasoning is correct. Have I misunderstood what your package will do?

[miketuri]

Hello @kthyng,

I have tried to provide more information for your questions below.

I'm concerned about this submission meeting the submission requirement of "The software should be a significant contribution to the available open source software that either enables some new research challenges to be addressed or makes addressing research challenges significantly better (e.g., faster, easier, simpler)." (https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements) if the problem your code addresses is not present in newer version of ngspice, which I suppose is what people are typically using, or at least have the ability to use if they want this functionality.

You are correct that newer versions (including the current version) of ngspice allow users to simulate parameter variations (or Monte Carlo simulations) of circuit netlists (note, there is a difference between the methods of performing these simulations with current ngspice and the proposed Perl scripts, but the results obtained would be similar).

I suppose the interpretation of "significant contribution" depends on the user: users that are able to work with current versions of ngspice would not necessarily need the proposed Perl scripts to run Monte Carlo simulations, however, users that need/want to work with a particular circuit or device model that is tied to an older version of ngspice would find this to be a "significant contribution." Without this Perl package, I would not have been able to complete parameter variation simulations for my FinFET circuit research (which would have prohibited me from publishing results in journals). This Perl package has also been used by a few of my colleagues who were engaged in similar research. I am unsure of what additional device or circuit models are tied to older versions of ngspice, but this proposed Perl package would likely be a "significant contribution" for users of similar models who want to (or need to) perform Monte Carlo simulations.

If my submission is still in-scope of JOSS, then I can provide self-testing information in the README file. Self-testing typically requires providing a circuit netlist and comparing the results/waveforms of the generated circuit netlist versus the original circuit netlist.

[kthyng]

You are correct that newer versions (including the current version) of ngspice allow users to simulate parameter variations (or Monte Carlo simulations) of circuit netlists (note, there is a difference between the methods of performing these simulations with current ngspice and the proposed Perl scripts, but the results obtained would be similar).

I suppose the interpretation of "significant contribution" depends on the user: users that are able to work with current versions of ngspice would not necessarily need the proposed Perl scripts to run Monte Carlo simulations, however, users that need/want to work with a particular circuit or device model that is tied to an older version of ngspice would find this to be a "significant contribution."

I guess this is what I don't understand — why would someone use an older version of the software instead of just updating?

[miketuri]

I guess this is what I don't understand — why would someone use an older version of the software instead of just updating?

It is not my choice to use an older version of ngspice--in a perfect world I would definitely prefer to use a more current version. However, the closed-source FinFET transistor model (see the references by Fossum et al.) that I use to model my circuits is built into an older version of ngspice without parameter variation/Monte Carlo simulation capabilities (I am only provided with the final binary to execute the simulator--I do not have source code for this). It is possible that I could instead use a different transistor model (this would allow me to use the current version of ngspice), however, this closed-source model is much more accurate than other available open-source models.

The proposed Perl script package can benefit other designers who wish to use this particular FinFET transistor model or can benefit designers who wish to use a different closed-source device model which is tied to an older version of Spice without parameter variation/Monte Carlo simulation support. I hope this information helps. Please let me know if you have any additional questions.

[kthyng]

@miketuri Ok, I am going to work on recruiting a topic editor who will know more about your field. I think some issues that could come up in review will include the packaging of your code (it looks like it could be more streamlined) and finding appropriate reviewers who use this code but older versions of it. Still, if such reviewers can be found, their input would be really valuable for your package!

[kthyng]

@whedon invite @katyhuff as editor

[whedon]

@katyhuff has been invited to edit this submission.

[kthyng]

@katyhuff Could you edit this submission? I am not sure who would be a better fit and your long term experience with JOSS and good code practices would be helpful here.

[katyhuff]

I'm afraid I'm totally slammed, not in the topic area (applied math for electrical circuit optimization, it seems), and behind, somewhat, on the other papers I'm currently editing for JOSS.

I could try, but I can't get to it for at least 3 weeks.

@miketuri sometimes the author has a better idea of who the handling editors should be for their own paper than anyone else. Perhaps you could take a look at this list of handling editors and let @kthyng and I know if there's someone who might fit your area a little better?

[miketuri]

Hello @kthyng and @katyhuff, I took a look at the list of editors and perhaps @labarba may be a good fit. This Perl script package works with ngspice (a version of Spice also used for mathematical modeling of circuits) and the Monte Carlo/parameter variation support offered by these scripts could be classified as applied math. Thank you

[labarba]

I am not able to edit this.

[kthyng]

@VivianePons would you look into editing this submission?

[VivianePons]

No problem

[danielskatz]

Thanks!

[danielskatz]

@whedon assign @VivianePons as editor

[whedon]

OK, the editor is @VivianePons

[danielskatz]

👋 @VivianePons - note that you can issue the command to whedon to assign a submission to yourself as well, you don't need to wait for an AEiC to do it :)

[VivianePons]

wave @VivianePons - note that you can issue the command to whedon to assign a submission to yourself as well, you don't need to wait for an AEiC to do it :)

Noted for next time!

[VivianePons]

Ok, I've just put myself up to date with the thread discussion. From what I understand, this package offers a useful (even essential) solution for people who have to use older versions of Spice and an alternative solution to the current implementation in Spice. Am I correct?

My main concern is that it seems very specialized. I want to make sure that it is usable by someone who is not you or one of your colleagues: does it have enough documentation? Testing? Is it robust enough that it works with slightly different configurations? Does it answer the need of a large enough community? Is it maintainable?

You don't have to answer to all these questions but you can comment if you want. My goal is to derive answers from what will come up in the review.

@miketuri do you have any recommendation for reviewers especially using our potential reviewer list

[miketuri]

Hello @VivianePons, thank you for taking a look at this. Your understanding is correct, this is essential for people using an older version of ngspice who want/need to perform parameter variations or Monte Carlo simulations (this package also allows people to take circuit measurements since this version of ngspice cannot--however, most waveform viewers can also do this). The transistor model I use for circuit design is not open-source and is compiled into an older version of ngspice; if it weren't for this, then I would have either used a current version of spice or tried to port the model to a current version of spice. This issue with models being compiled into a version of spice can happen and cause these issues where certain spice functionality is not present.

I took a look at the potential reviewer list. Here are my recommendations (usernames follow): fpl, pragyansmita, Brnikat, earino

Please let me know if you need any additional information or more suggestions for reviewers. Thank you.

[mzszym]

@VivianePons I think I could be a reviewer. I unfortunately never used Perl, so I will not be able to comment much on the internal code, but I am familiar with SPICE and circuit simulations, and in general electronic simulation tools, and I think I could be able to review this according to the guidelines. Also, I think there is a merit in creating external tools for SPICE, just because SPICE (ngspice) is a very old code which is hard to extend and maintain, and would profit from a refactor which no one has resources to do. I think, even if an external tool mimics functionality in ngspice to some extent, the possibility to adapt and modify it in more easily that ngspice code is itself meriting.

[VivianePons]

@mzszym Thank you, I asked you indeed because I saw you had some knowledge of SPICE so I figured you would be in a good place to evaluate how practicable the software is. In my opinion (I could be wrong!) most potential users will know SPICE but not always Pearl.

Thank you for your input by the way on creating an external tool for SPICE.

Also know that I am looking for a second reviewer who will know Pearl (not necessary SPICE unless I find somebody who know both!)

[VivianePons]

@whedon assign @mzszym as reviewer

[whedon]

OK, @mzszym is now a reviewer

[VivianePons]

@pragyansmita would you be willing to review this submission?

[VivianePons]

Dear @miketuri I have difficulties finding a second reviewer for your submission. Especially, it seems difficult to find someone who is familiar with SPICE (which seems important) and also knows Perl. would you have any suggestions, maybe outside of our potential referees list?

[miketuri]

Dear @VivianePons I have reached out to a contact to see if he, or someone he knows, would be able to review this software. I'll keep you posted with what I find out.

[VivianePons]

@whedon check repository

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.14 s (91.8 files/s, 37051.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Perl                             9            463           1191           2439
Markdown                         2            203              0            682
Tcl/Tk                           1             33             68            105
TeX                              1              7              0             57
-------------------------------------------------------------------------------
SUM:                            13            706           1259           3283
-------------------------------------------------------------------------------

Statistical information for the repository '1988' was gathered on 2020/03/12.
No commited files with the specified extensions were found.

[arfon]

Dear authors and reviewers

We wanted to notify you that in light of the current COVID-19 pandemic, JOSS has decided to suspend submission of new manuscripts and to handle existing manuscripts (such as this one) on a "best efforts basis". We understand that you may need to attend to more pressing issues than completing a review or updating a repository in response to a review. If this is the case, a quick note indicating that you need to put a "pause" on your involvement with a review would be appreciated but is not required.

Thanks in advance for your understanding.

Arfon Smith, Editor in Chief, on behalf of the JOSS editorial team.

[miketuri]

Dear @VivianePons I hope all is well with you. I had reached out to a contact, however he and his team members are unable to review this package (not familiar with Perl). I am looking at previous JOSS submissions to see if it looks like any Perl software author also has Spice knowledge and would be willing to assist with this review. I'll keep you posted with what I find out.