elise1993
commented
5 months ago Effect of Stackmax and Rmax
Using the following conditions (see /tests/test1a.mat for an exhaustive list)
| System | Multi-Step Prediction | $dt$ |
Interpolation Factor |
|---|---|---|---|
| Lorenz System | 100 | 0.01 | 0.1 |
The prediction accuracy naturally increases with increasing stackmax and r, until it reaches the tolerance set for the ODE solver (1e-12):
| stackmax = rmax | r | RMSE | NMSE | $R^2$ |
|---|---|---|---|---|
| 6 | 3 | 8.97e-1 | 1.30e-2 | 0.99 |
| 10 | 5 | 4.57e-1 | 3.38e-3 | 1.00 |
| 40 | 18 | 3.46e-2 | 1.94e-5 | 1.00 |
| 100 | 47 | 9.52e-5 | 1.47e-10 | 1.00 |
| 400 | 194 | 2.04e-5 | 6.76e-12 | 1.00 |
| 1000 | 491 | 2.07e-5 | 7.06e-12 | 1.00 |
Performance on Various Systems with Linear Model (HAVOK-SINDy)
Here we test the accuracy of the linearization for various systems, i.e. we only forecast using $\dot{\textbf{v}}=\textbf{Av}$. Stackmax and rmax have been adjusted somewhat for each system to obtain a better separation of linear and nonlinear dynamics.
With similar conditions as above (see /tests/test1b.mat for exhaustive lists):
| System | stackmax | rmax | RMSE | NMSE | $R^2$ |
|---|---|---|---|---|---|
| Lorenz System | 39 | 7 | 6.35e-2 | 6.53e-5 | 1.00 |
| Rössler System | 100 | 6 | 3.37e-2 | 8.37e-6 | 1.00 |
| Van der Pol System | 39 | 7 | 7.16e-4 | 2.12e-7 | 1.00 |
| Duffing Oscillator | 50 | 7 | 8.06e-6 | 1.47e-11 | 1.00 |
| Double Pendulum | 40 | 10 | 3.62e-3 | 1.62e-5 | 1.00 |
| Mackey-Glass System | 40 | 5 | 5.60e-5 | 5.99e-8 | 1.00 |
| Magnetic Field Reversal | 100 | 4 | 9.40e2 | 2.84e-2 | 0.97 |
Lorenz system
Rossler system
Van der Pol system
Duffing oscillator
Double pendulum
Mackey-Glass system
Magnetic Field Reversal
Performance on Various Systems with Forced Linear Model (HAVOK-SINDy-ML)
Ideally one would choose the ML method that best fits the type of system, but here we have simply chosen to use Random Forest Regression (RFR) for all systems. As can be seen for the Magnetic Field Reversal case, RFR is not able to predict the chaotic jumps, and thus performs poorly in this case.
| Multi-Step Prediction | Regressor |
|---|---|
| 1000 | RFR-MEX |
| System | stackmax | rmax | RMSE | NMSE | $R^2$ |
|---|---|---|---|---|---|
| Lorenz System | 39 | 7 | 6.15e+0 | 6.13e-1 | 0.57 |
| Rössler System | 100 | 6 | 5.46e+0 | 2.20e-1 | 0.81 |
| Van der Pol System | 39 | 7 | 6.51e-2 | 1.75e-3 | 1.00 |
| Duffing Oscillator | 50 | 7 | 2.96e-2 | 1.99e-4 | 1.00 |
| Double Pendulum | 40 | 10 | 8.21e-1 | 8.33e-1 | 0.42 |
| Mackey-Glass System | 40 | 5 | 4.97e-2 | 4.72e-2 | 0.96 |
| Magnetic Field Reversal | 100 | 4 | 5.71e+3 | 1.05e+0 | 0.17 |
Lorenz system
Rossler system
Van der Pol system
Duffing oscillator
Double pendulum
Mackey-Glass system
Magnetic Field Reversal
To verify that the code works as intended, we have a suite of test cases representing different types of dynamical systems, including Ordinary Differential Equations (ODEs), Delay Differential Equations (DDEs), and Stochastic Differential Equations (SDEs):
t_0:dt:t_{max}$\alpha=10, \rho=28, \beta=8/3$ | $[-8, 8, 27]$a=0.1, b=0.1, c=14$ | $[1,1,1]$\mu=5$ | $[1,1]$\delta=0, \alpha=1, \beta=4$ | $[1,1]$l_1=l_2=m_1=m_2=1, g=10$ | $[\pi/2,-0.01,$ $\pi/2,-0.005]$\beta=2, \tau=2, n=9.65, \gamma=1$ | $1$\gamma=[1.5,1.5], \epsilon=[1.5,0.6], \tau=1, K=0.05,$ $V=[0.0017068, 0, -0.1536767, 0, 0], w=0.2$ | $6.7096 [1, 1]$To verify the correctness of HAVOK-SINDy and test its performance, we initially exclude the ML term, as the accuracy of the ML model is highly dependent on the type of system and ML parameters such as tree size, number of layers, etc. Thus we construct the HAVOK-SINDy model, and during forecasting we feed the true intermittent forcing into this model for testing.
We use the following performance metrics to test the accuracy of the HAVOK-SINDy method:
Root Mean Square Error (RMSE): $
= \sqrt{ \frac{1}{n} \sum\limits_{i=1}^{n} \left( x_i - \hat{x}_i \right)^2 }$Normalized Mean Square Error (NMSE): $
= \frac{ \sum\limits_{i=1}^{n} (x_i - \hat{x}_i)^2 }{ \sum\limits_{i=1}^{n} (x_i - \bar{x})^2 }$$R^2$ error: $
= \frac{ \left(\sum\limits_{i=1}^{n}(x_i-\bar{x})(\hat{x}_i-\bar{\hat{x}}) \right)^2 }{ \sum\limits_{i=1}^{n}(x_i-\bar{x})^2 \sum\limits_{i=1}^{n}(\hat{x}_i-\bar{\hat{x}})^2}$We expect these performance metrics for the HAVOK-SINDy model to asymptotically approach zero as stackmax and rmax increases.