[PRE REVIEW]: SysIdentPy: A Python package for System Identification

[whedon]

Submitting author: @wilsonrljr (Wilson Rocha Lacerda Junior) Repository: https://github.com/wilsonrljr/sysidentpy Version: 0.1.1 Editor: @dpsanders Reviewers: @Shibabrat, @dawbarton Managing EiC: Arfon Smith

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @wilsonrljr. Currently, there isn't an JOSS editor assigned to your paper.

The author's suggestion for the handling editor is @dpsanders.

@wilsonrljr if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- None

MISSING DOIs

- https://doi.org/10.1016/0005-1098(89)90019-8 may be missing for title: System identification: theory for the user
- https://doi.org/10.1080/00207178908559683 may be missing for title: Representations of non-linear systems: the NARMAX model
- https://doi.org/10.1016/j.ymssp.2015.12.031 may be missing for title: Sufficient conditions for rate-independent hysteresis in autoregressive identified models
- https://doi.org/10.1109/access.2019.2926057 may be missing for title: Control of Hysteretic Systems Through an Analytical Inverse Compensation Based on a NARX Model
- https://doi.org/10.1016/s0307-904x(03)00071-4 may be missing for title: Modelling and prediction of machining errors using ARMAX and NARMAX structures
- https://doi.org/10.1016/j.neucom.2015.08.022 may be missing for title: Ultra-orthogonal forward regression algorithms for the identification of non-linear dynamic systems
- https://doi.org/10.1109/tbme.2002.803507 may be missing for title: NARMAX representation and identification of ankle dynamics
- https://doi.org/10.1016/b978-0-12-811788-0.00008-1 may be missing for title: Applications of NARMAX in Space Weather

INVALID DOIs

- None

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[arfon]

:wave: @dpsanders - the submitting author suggested you as the handling editor for this one. I'll go ahead and assign this to you now - please let me know if this is a problem.

[arfon]

@whedon assign @dpsanders as editor

[whedon]

OK, the editor is @dpsanders

[dpsanders]

@arfon OK sure.

[wilsonrljr]

Suggestions for potential reviewers:

• sixpearls
• ixjlyons
• shibabrat
• dawbarton

Based on the spreadsheet provided, the reviewers mentioned above are experts in nonlinear dynamic systems, data science, signal processing , and many correlated areas to the topic of the paper.

[dpsanders]

👋 Hi @wilsonrljr. Many thanks for the list of possible reviewers. I hope we can get the review going soon, subject of course to people's availability.

[wilsonrljr]

You're welcome! I am available to help in whatever way possible throughout the process.

[dpsanders]

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Re-invite a reviewer (if they can't update checklists)
@whedon re-invite @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Remind an author or reviewer to return to a review after a
# certain period of time (supported units days and weeks)
@whedon remind @reviewer in 2 weeks

# Ask Whedon to do a  dry run of accepting the paper and depositing with Crossref
@whedon accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

EiC TASKS

# Invite an editor to edit a submission (sending them an email)
@whedon invite @editor as editor

# Reject a paper
@whedon reject

# Withdraw a paper
@whedon withdraw

# Ask Whedon to actually accept the paper and deposit with Crossref
@whedon accept deposit=true

[dpsanders]

@whedon check repository

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.83 s (129.4 files/s, 83551.7 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
SVG                             15              0            313          16301
HTML                            23           3702             23           9694
JavaScript                      12           2224           2264           8467
Python                          21            677           1358           1599
Jupyter Notebook                17              0          20281            928
CSS                              6            167             34            590
reStructuredText                 8            244            198            256
Markdown                         2             61              0            161
TeX                              1             13              0            120
DOS Batch                        1              8              1             26
make                             1              5              7              9
YAML                             1              0              0              1
-------------------------------------------------------------------------------
SUM:                           108           7101          24479          38152
-------------------------------------------------------------------------------

Statistical information for the repository '2191' was gathered on 2020/05/28.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
LuanPascoal                      6            95             96            0.27
Samuel                           1            87             35            0.17
Samuel Oliveira                 14           721            409            1.58
Wilson Rocha                     1           366              0            0.51
renard162                        3            71             31            0.14
wilsonrljr                      36         41884          27668           97.33

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
wilsonrljr                16590           39.6          0.2               15.02

[dpsanders]

Hi @wilsonrljr, sorry for the delay.

I have a few questions and comments from looking at your submission:

• If this is a package for doing NARMAX, should NARMAX appear in the name of the package?
• How does this compare to something like PySindy?
• Since this seems to be a way of inferring a model from data, how is it related to e.g. Scikit-Learn?

About the paper:

• You use F sometimes and ℱ (script F) other times.
• Perhaps there's a way of abbreviating equation (2)?
• It is not clear what type of objects the Θ's are.
• It is not clear where the degree ℓ comes into the equation.
• It's not clear why it's relevant that polynomial functions are smooth and the Weierstrass theorem.
• You don't say what SISO and MISO, AIC etc. are. Maybe that can be left to the documentation, but you should say that.
• provid -> provide
• It is not clear what the table and figure are telling me. I.e. what is it exactly that your package outputs and how can I use that?

Basically I need your paper to explain what the package provides for a non-expert in the field (me).

Thanks!

[wilsonrljr]

Hi @dpsanders ! Thank you for your comments and efforts towards improving our manuscript. In the following I will address all your questions and suggestions.

• If this is a package for doing NARMAX, should NARMAX appear in the name of the package? In fact, adding NARMAX to the title would be a good way to make it clearer what the package is about.

• How does this compare to something like PySindy? I take a look at PySindy and it is a great package for System Identification. However, although SysIdentPy and PySindy belong to the same field, they are very different model class. PySindy deals with autonomous systems like Lorenz System and other chaotic systems based on numerical integration and differentiation methods for optimization.

On the other hand, SysIdentPy is based on nonlinear autoregressive models with multiple inputs and outputs. To build the models we rely on algorithms like Forward Orthogonal Least Squares Algorithms and Error Reduction Ratio. The core of NARMAX models is the model structure selection do select the correct terms using methods such as the aforementioned algorithms. At this point we have polynomial NARMAX (the most used basis function for NARMAX), but we already implementing Neural NARMAX, for example. In short, SysIdenPy and PySindy provide very different ways to build dynamical models based on different goals, assumptions and methods.

• Since this seems to be a way of inferring a model from data, how is it related to e.g. Scikit-Learn? SysIdenPy provides built-in and native methods for building nonlinear autoregressive models using traditional algorithms in System Identification field. For modeling dynamical data using parametric models, some specific methods are required. Scikit-Learn have useful tools for building models from static data since but they not support dynamical data modeling. NARMAX models are the most used model class for building dynamical models. SysIdentPy offer native solutions and algorithms regarding NARMAX models including the model structure selection algorithms, the model order selection algorithms, several methods for parameter estimation and many other features.

  Actually, SysIdentPy have more in common with StatsModels package in terms of models class. But statsmodels only support linear autregressive models (ARMAX). Nonlinear models requires robust model structure selection and parameter estimation algorithms, whice are provided in SysIdentPy.

In addition, SysIdentPy package is designed to be accessible to scikit-learn users while advanced users can customize it in many ways for different analysis.

• You use F sometimes and ℱ (script F) other times. Thank you. The correction has been made.

• Perhaps there's a way of abbreviating equation (2)? We changed it to a shortened version of the equation.

• It is not clear what type of objects the Θ's are.

• It is not clear where the degree ℓ comes into the equation. We added an example showing what are the $\Theta$ and $\ell$. Actually, we removed $\ell$ from equation 1 to use $\mathcal{F}$ to represent any function and we explain the polynomial NARMAX with more detail from equation 2 to make it more easy to understand.

• It's not clear why it's relevant that polynomial functions are smooth and the Weierstrass theorem. The relevance relies on the fact that polynomials functions can be used to represent several nonlinear systems. As we mentioned in the paper, NARMAX models can be represented in a polynomial basis, neural basis, wavelet and so on. Although we are already implementing extensions like neural NARMAX, we like to highlight those polynomial NARMAX features because System Identification is not just about prediction, but can be also used for analysis of stability and control of nonlinear systems both in discrete and continuous time. In this respect, those kind of features make polynomial NARMAX powerful alternatives to for example neural, rbf NARMAX because you can easily access the model structure, take the derivatives, obtain an inverse model analytically and so on.

• You don't say what SISO and MISO, AIC etc. are. Maybe that can be left to the documentation, but you should say that. Thank you for point out. We added more information in the respective paragraph as well some references for make it more clear. Also, we advise the user to read the package documentation for a more in depth details of those methods.

• provid -> provide The correction has been made.

• It is not clear what the table and figure are telling me. I.e. what is it exactly that your package outputs and how can I use that? Thank you for this comment. We rewrote the paragraph explaining in more detail the procedure presented.

Note: Due to the mentioned changes, we have also have adapted some small parts here and there in the paper. If you need me to specify all the small changes, let me know.

If you have more suggestions, just tell me!

[dpsanders]

@whedon generate pdf

[whedon]

PDF failed to compile for issue #2191 with the following error:

Error producing PDF. ! Package amsmath Error: Multiple \label's: label 'eq:narxekxt' will be lost.

See the amsmath package documentation for explanation. Type H for immediate help. ...

l.409 \end{align}

Looks like we failed to compile the PDF

[dpsanders]

@wilsonrljr Sorry about the delay. Could you please fix the compilation error here?

[wilsonrljr]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[wilsonrljr]

@whedon generate pdf

[whedon]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@whedon commands

[wilsonrljr]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[wilsonrljr]

No problems, @dpsanders . The error must be fixed now. I also fixed two symbols (missing backslash) and removed two repeated words on paper.

[dpsanders]

👋 Hi @Shibabrat and @dawbarton. Would either of you be willing and able to review this submission for JOSS? We carry out our checklist-driven reviews here in GitHub issues and follow these guidelines: https://joss.readthedocs.io/en/latest/review_criteria.html. Thanks!

[dawbarton]

Yes, I'd be willing to. I've worked a little with NARMAX models in the past.

[Shibabrat]

Hi @dpsanders, yes, I'm willing to review as a non-expert in the system identification who works with nonlinear dynamical systems in general.

[dpsanders]

@whedon assign @dawbarton as reviewer

[whedon]

OK, @dawbarton is now a reviewer

[dpsanders]

@whedon assign @Shibabrat as reviewer

[whedon]

OK, @Shibabrat is now a reviewer

[dpsanders]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/2384.

[dpsanders]

Thanks @dawbarton and @Shibabrat for agreeing to review this submission!

[dpsanders]

@whedon add @dawbarton as reviewer

[whedon]

OK, @dawbarton is now a reviewer

[dpsanders]

@whedon commands

[whedon]

Here are some things you can ask me to do:

# List all of Whedon's capabilities
@whedon commands

# Assign a GitHub user as the sole reviewer of this submission
@whedon assign @username as reviewer

# Add a GitHub user to the reviewers of this submission
@whedon add @username as reviewer

# Re-invite a reviewer (if they can't update checklists)
@whedon re-invite @username as reviewer

# Remove a GitHub user from the reviewers of this submission
@whedon remove @username as reviewer

# List of editor GitHub usernames
@whedon list editors

# List of reviewers together with programming language preferences and domain expertise
@whedon list reviewers

# Change editorial assignment
@whedon assign @username as editor

# Set the software archive DOI at the top of the issue e.g.
@whedon set 10.0000/zenodo.00000 as archive

# Set the software version at the top of the issue e.g.
@whedon set v1.0.1 as version

# Open the review issue
@whedon start review

EDITORIAL TASKS

# Compile the paper
@whedon generate pdf

# Compile the paper from alternative branch
@whedon generate pdf from branch custom-branch-name

# Remind an author or reviewer to return to a review after a
# certain period of time (supported units days and weeks)
@whedon remind @reviewer in 2 weeks

# Ask Whedon to do a  dry run of accepting the paper and depositing with Crossref
@whedon accept

# Ask Whedon to check the references for missing DOIs
@whedon check references

# Ask Whedon to check repository statistics for the submitted software
@whedon check repository

EiC TASKS

# Invite an editor to edit a submission (sending them an email)
@whedon invite @editor as editor

# Reject a paper
@whedon reject

# Withdraw a paper
@whedon withdraw

# Ask Whedon to actually accept the paper and deposit with Crossref
@whedon accept deposit=true

[dpsanders]

@whedon list reviewers

[whedon]

Here's the current list of reviewers: https://bit.ly/joss-reviewers

[dpsanders]

@whedon start review

[arfon]

@whedon start review

@dpsanders - I've just responded over in #2384 - you'll need to fix up the review checklists over there.