Output of the identified model Yid is not correct

[YenyAr]

Hi! I am fitting an ARMAX model for a MISO system with the following orders: na = 3; nb = [3,3]; nc = [2]; nk=[0,0]. The coefficients of A, B, C that I get are very similar to the ones I get in matlab. However the Yid output I get is very close to the original, with an R^2 of 99.99% and does not make sense. The R^2 I get with the matlab output is 80%. When I set na = 0, it seems that the result makes sense, and when I set any value to na, the fitted output matches the original output.

[RBdC]

Dear YenyAr, thank you for using SIPPY.. ... about your issue, it might be that you do not have a real problem.

It is surely a positive stuff that your coefficients are close to the one obtained in Matlab, but an R^2 of 99.99% may have sense too. Did you include noise in the data used for the identification? If no noise is added, a very high value of R can be obtained, especially when the model orders are appropriate, a very high match of the original output can be achieved.

Using na = 0 means excluding the Autoregressive component of your ARMAX model, that is, your model is simpler, maybe too simple, so that, the performance are surely lower

Moreover, which algorithm did you use? Iterative Least Squares, Recursive LS o Optimization-based? I would guess you used I-LS, which is proved to give the best performance, in our experience

Best Sippy Teams

[YenyAr]

Hello! thank you for your reply. I am modeling with real data. My output are natural frequencies measured in a structure and my inputs are environmental variables like temperature. Because of the nature of the data there is a high noise content, so I am concerned that I am getting 99.99 %. Likewise, if I run the regression with frequencies from another structure that I know have even more noise content I still get 99.99%.

inputs.txt output.txt

na = [3] nb = [[3,3]] nc = [2] nk = [[0,0]]

Id_ARMAXi = system_identification(output, inputs, 'ARMAX', ARMAX_orders=[na, nb, nc, nk], centering = 'MeanVal')

model_output = Id_ARMAXi.Yid

[RBdC]

Dear YenAr, your issue is now much more clear. You are right, the results you report appear actually pretty strange; two same values of the output fitting for two different levels of noise (with all same conditions/parameters) seems illogical.

Thank you for your data; May I ask to share also your example .py code (by mail, if you prefer - riccardo.bacci@unipi.it)? so that, we can test it quickly and investigate eventual bugs/errors within SIPPY. All the best SIPPY Team

[YenyAr]

Thank you RBdC. I sent you the e-mail.

[RBdC]

Dear YenyAr,

thank for your mail. We did analyze your case and files. Everything proves good with SIPPY and, of course, with the sys-id MATLAB toolbox.

In attach you find an updated .py file (from your) and a corresponding .m file.

An ARMAX (3,[1,1],1) model is identified, and the performance in terms of identification fitting (that is, R2) and also in terms of validation analysis (1-step, 10-step ahead, and pure open-loop simulation) are provided.

Your data seems pretty easy to be identified and results from SIPPY and MATLAB are actually very aligned.

As said, when using na = 0 means excluding the Autoregressive component of your ARMAX model, that is, your model is simpler, maybe too simple, so that, the performance are surely lower.

So you can be very confident with the results!

Hoping all is clear, all the best SIPPY Team

ARMAX_frequencies_ST.py.zip

m_test.m.zip

[YenyAr]

Thank you!!