Paper gameplan

[mdmurbach]

🎉 The impedance.py project is now over a year and a half old 🎉 (first commit was in May 2018) and it's turned into quite a useful tool (I think I've had someone come up to me saying they've found it to be useful or requesting a feature at pretty much every conference/meeting I've been at recently!). In order to further promote impedance.py and give folks some way of documenting their use of the software, I'd like to propose we write and submit a paper. The combination of low-overhead and open-focus of the Journal of Open Source Software (JOSS) seem to make sense to me (although other suggestions would be welcome). The submission essentially requires a short paper and some well-documented/organized/useful code.

Authorship 📚

I think anyone who has contributed significantly to impedance.py should be an author. Here is my current proposal, from the contributions GitHub page (but feel free to reach out on GitHub/email/slack if you have other thoughts):

Matt Murbach -- @mdmurbach Brian Gerwe -- @BGerwe Neal Dawson-Elli-- @nealde Lok-kun Tsui -- @lktsui

Remaining Todos 🏷

I've created a project (https://github.com/ECSHackWeek/impedance.py/projects/1) for tracking what we need to take care of before submitting the paper. Feel free to add anything to the list or tackle some of the remaining issues. Big ones in my mind are the current refactor, the linKK issues @BGerwe has found, improved docs/error checking/handling, ...

@dacb @dt-schwartz any experience with JOSS/advice for anything else we should be thinking about when submitting a paper?

[dt-schwartz]

No experience on my part. I'd note that it is a good idea to publish it. I've seen 1 Google Scholar citation to impedance.py in a paper, referencing Matt's ECS Interface as the source.

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Matt Murbach notifications@github.com Sent: Tuesday, February 18, 2020, 8:41 PM To: ECSHackWeek/impedance.py Cc: Dan Schwartz; Mention Subject: [ECSHackWeek/impedance.py] Paper gameplan (#80)

🎉 The impedance.py project is now over a year and a half old 🎉 (first commit was in May 2018) and it's turned into quite a useful tool (I think I've had someone come up to me saying they've found it to be useful or requesting a feature at pretty much every conference/meeting I've been at recently!). In order to further promote impedance.py and give folks some way of documenting their use of the software, I'd like to propose we write and submit a paper. The combination of low-overhead and open-focus of the Journal of Open Source Software (JOSS)https://joss.theoj.org/ seem to make sense to me (although other suggestions would be welcome). The submission essentially requireshttps://joss.readthedocs.io/en/latest/submitting.html a short paper and some well-documented/organized/useful code.

Authorship 📚

I think anyone who has contributed significantly to impedance.py should be an author. Here is my current proposal, from the contributionshttps://github.com/ECSHackWeek/impedance.py/graphs/contributors GitHub page (but feel free to reach out on GitHub/email/slack if you have other thoughts):

Matt Murbach -- @mdmurbachhttps://github.com/mdmurbach Brian Gerwe -- @BGerwehttps://github.com/BGerwe Neal Dawson-Elli-- @nealdehttps://github.com/nealde Lok-kun Tsui -- @lktsuihttps://github.com/lktsui

Remaining Todos 🏷

I've created a project (https://github.com/ECSHackWeek/impedance.py/projects/1) for tracking what we need to take care of before submitting the paper. Feel free to add anything to the list or tackle some of the remaining issues. Big ones in my mind are the current refactor, the linKK issues @BGerwehttps://github.com/BGerwe has found, improved docs/error checking/handling, ...

@dacbhttps://github.com/dacb @dt-schwartzhttps://github.com/dt-schwartz any experience with JOSS/advice for anything else we should be thinking about when submitting a paper?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ECSHackWeek/impedance.py/issues/80?email_source=notifications&email_token=AHWZ73A3HFQVKX4HIKVJVJLRDSZ6DA5CNFSM4KXRT2ZKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IOQPSTA, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHWZ73B75PE3K3FBGU4EXFTRDSZ6DANCNFSM4KXRT2ZA.

[lktsui]

I am having some trouble trying to figure out JOSS. It looks like most submissions are no more than one page long?

Throwing it out there since this is the first thing that comes to mind for where to send this work:

Would we also want to consider a journal more specific to electrochemistry like Journal of the Electrochemical Society? Maybe a short <1 page submission over to JOSS and a more in-depth writeup on our algorithms and applications of impedance.py to some system of interest to JECS?

[dt-schwartz]

Hi Lok-kun,

My take on the decision and submission process for the major contributors to impedance.py would be to consider a heirarchy of goals and priorities:

(1) get impedance.py recognized as a valuable research product.

A JOSS "paper" has undergone peer reviewed, so it goes into that part of your CV, has a long-lived and archived digital object attached, so can be a ligitimate part of your federal grant CV, and because it is a "journal" old guys like me don't get confused.

A JOSS paper says, "embedding the knowledge of a community into a useable and we'll documented piece of software IS scholarship."

I have been watching your exchanges about the best way to do this or that, and I am convinced impedance.py is a scholarly product embedding knowledge and wisdom of well educated electrochemist/engineers. Think of it as a "review paper" built into software.

(2) generate new insights with impedance.py

I think you could get new data or (better yet) reuse published data from others and show off the features and value of this approach (for example) by quickly comparing models, showing confidence bounds, etc.. When you learn something, report it in a conventional (i.e., JES) journal article.

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: Lok-kun Tsui notifications@github.com Sent: Tuesday, February 18, 2020, 10:48 PM To: ECSHackWeek/impedance.py Cc: Dan Schwartz; Mention Subject: Re: [ECSHackWeek/impedance.py] Paper gameplan (#80)

I am having some trouble trying to figure out JOSS. It looks like most submissions are no more than one page long?

Throwing it out there since this is the first thing that comes to mind for where to send this work:

Would we also want to consider a journal more specific to electrochemistry like Journal of the Electrochemical Society? Maybe a short <1 page submission over to JOSS and a more in-depth writeup on our algorithms and applications of impedance.py to some system of interest to JECS?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ECSHackWeek/impedance.py/issues/80?email_source=notifications&email_token=AHWZ73A7O52IRBQJRLPCCPTRDTI2PA5CNFSM4KXRT2ZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMGSCHQ#issuecomment-588063006, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHWZ73HH7YKAQ6A72ZJMVITRDTI2PANCNFSM4KXRT2ZA.

[mdmurbach]

@lktsui and @dt-schwartz I could see it make sense to think about this as a two-step process where we (i) submit a quick write up to JOSS and (ii) start putting together a more traditional paper. This i/ii combo is actually common enough in other fields that JOSS has partnered with traditional astronomy journals (AAS), so there are some prior examples out there of this working.

For the more conventional manuscript, I agree with Dan that it would be interesting to reuse published data. In terms of content, what comes to mind off the top of my head is: analyze previously published data in a reproducible way with impedance.py (highlighting how easy-to-use software from the 21st century can drive community adoption of the best practices of validation, visualization, thinking of the 20th century), show that model selection is important (via confidence intervals, maybe F-tests, -- both of which would be new to EIS analysis I believe). I also think direct comparisons of DRT/equivalent circuits/continuum models would be an interersting future direction, but that requires a fair amount of new development.

I'd be curious to get your thoughts on what a more conventional paper could look like @lktsui. @BGerwe, yours as well since we've talked a bit about future directions of the project.

[BGerwe]

I agree with @mdmurbach we should prioritize a quick JOSS paper. After looking through the JOSS review criteria, we're well on track! With ECS's Open Science initiatives I wonder if there is a chance to catalyze collaboration between ECS and JOSS.

For a more conventional paper, I'm skeptical the package represents a significant technical contribution on its own. Pairing it with quantitative model discrimination (like confidence intervals of F-tests as you mentioned) should be significant enough. I agree with reusing published data – collecting new data untenably expands the scope of this work – and we should focus on data with "controversial" analysis where workers disagreed on model selection for some time before settling on a model. Hopefully, we can successfully select the correct model with our model discrimination methods. Ideally, we could touch on several fields (batteries, fuel cells, corrosion, etc) to show the generality of the package. Maybe @dt-schwartz can point us to some examples?

[dt-schwartz]

Matt has data for Li-ion batteries that can be reused and also agree it would be great for you to use non-battery data as well. I think a JOSS/ECS partnership would be valuable, but would decouple your progress from that.

For JES, the software package, in and of itself, is not the topic of the paper, though I am not sure you need to set the bar at "solve a controversy". Instead, think of it as a mini review of model-data convergence in impedance spectroscopy, emphasizing the need for better statistical tools for understanding confidence and model discrimination, then show-off the ease of using this tool to answer some questions about model-data convergence that you've not seen before.

Definitely do some model discrimination of real models people might use for a data set, and compare say with F-test. FOr Matt's data, this might be using Patsch's model+ warburgh vs constant phase approach. There might be ways to show off other aspects that people have not thought too much about. For example, the paper (S. Fletcher, J. Electrochem. Soc., 141, 1823 (1994)) shows that ECM models can have exactly identical frequency responses using totally different circuit elements (degeneracy). That means, the best fit for different degenerate models ought to give exactly the same sum of square errors for a real dataset with noise. But the same fits may have different parameter confidence intervals and confidence envelope (most interesting for models with identical number of fit parameters). This would perhaps let you illustrate some software features, by showing multiple models with exactly the same fit can have different uncertainty traits, driven by way frequency-dependent experimental noise interacts with the model parameters. You could show some of that with real experimental data and synthetic data having different noise traits. Just some ideas, with the main one idea here being...set a reasonable bar for the new way to think about model-data convergence enabled by the software since you all have day jobs!

[mdmurbach]

Made a first pass at a paper.md file that follows the JOSS guidelines. Feel free to edit/expand the paper as you see fit. Also, please make sure you add your orcid/email/affiliation as well.

You can find the source file in joss-paper/paper.md on the paper/joss branch and a compiled version via the JOSS service here.

If you make edits and want to see how it compiles you can use the JOSS whedon service: https://whedon.theoj.org/

[dt-schwartz]

The paper looking great. Nice job driving to v.1 release and draft paper. It has been fun to watch on GitHub. What is the dataset you fit?

[mdmurbach]

Thanks, Dan. And good catch, the data is from a lithium-ion battery from the NLEIS experimental paper. I should add a citation to the dataset.

[BGerwe]

This looks like a great start! After looking at a few recently published JOSS papers, I think the "Getting Started" section is a bit too granular. We could also give a bit more background on using equivalent circuits to interpret EIS experiments and tie this in with a comparison of other freely available software (PyEIS at least) to highlight the flexibility of defining custom circuits and elements.

I'll try getting to that this afternoon but for now I added my orcid and fixed a couple typos.

[lktsui]

I can't seem to figure out how to use the PDF builder. It gives me a "Not Found" error.

I have a few suggestions:

• A list of supported data formats currently implemented so people can know if their potentiostat is compatible with our program.
• A list of supported circuit elements. Some impedance analysis software I've encountered only support basic L, C, R responses and maybe a CPE. We could highlight our support for some of the more electrochemistry specific circuit elements.
• In the example after the fit is performed, print out the custom circuit's fitted parameters to show what information is obtained after a fit.
• I would like to see a couple sentences on how the fitting works and what underlying math is used could be added to the background. Something like:

Each model generates a frequency dependent impedance response. The SciPy optimization package using the Levenberg-Marquardt algorithm (I think this is the default?) is used to minimize the error between the experimental impedance and the model's impedance.

• Stray ' in Figure 1 caption?

[mdmurbach]

These are great suggestions @lktsui and @BGerwe.

@all I made some more edits to add more explanation of the features. Not sure if the organization makes sense or if it's too marketing-y (I spent the day thinking about customer/investor pitches...), but I agree with you both that it's a good idea to highlight features of the package.

[BGerwe]

@ALL I restructured the top section, breaking it into a background on EIS (I felt that was needed since the journal audience is broad) and an explicit statement of need (per the JOSS submission guidelines). Should we change the functionality headers to reflect the current module names? Or vice-versa? Either way, we should have consistent naming between the package and the paper.

[mdmurbach]

@BGerwe, great thanks! I made a few comments on your commit, looks great overall! In terms of the consistent naming what do you think of having the titles as "Data Ingestion (impedance.preprocessing)", "Data Validation (impedance.validation)", etc.?

[lktsui]

Do we want to just call the Data Injestion title "Data Preprocessing"?

[mdmurbach]

Now that's thinking @lktsui! :tada: yeah, that's a really good and much cleaner/simpler idea

[lktsui]

Matt and I discussed adding a statement about how our software is cross platform while other options often don't support Mac or Linux, so I have added a statement in the above commit to the Statement of Need section.

Added that we now support CH Instruments data format in - https://github.com/lktsui/impedance.py/commit/abc4c8536a07601b46a9db5f246d6d724943b4f6

[BGerwe]

Matt, I think you deserve credit that "Data Ingestion" is a sexy marketing, but it's a little incongruent with how the package is set up.

[dt-schwartz]

Your JOSS paper is looking good. To be more physical for the battery folks, the circuit element you define, fit, and plot may want to move the Warburg impedance inside the slower parallel Randal-like elements (in series with a charge transfer resistance). It will probably make a small difference in the fit values.