search

dynamicslab / pysindy

A package for the sparse identification of nonlinear dynamical systems from data
https://pysindy.readthedocs.io/en/latest/
Other
1.36k stars 304 forks source link

[BUG] A lot of issues with the bayesian implementation (UQ-SINDy) #517

Closed ggmirandac closed 3 weeks ago

ggmirandac commented 1 month ago

Hi,

I am trying to use the SBR optimizer in my pipeline to enhance the pipeline by allowing for uncertainty quantification.

To do so, I follow the steps at #496. And use the following shell command to install the version with the SBR optimizer:

pip3 install "pysindy[sbr]@git+https://github.com/dynamicslab/pysindy"

Following that, I try to use the SBR optimizer and got the following: Code:

import numpy as np
import pysindy as ps

t = np.linspace(0, 1, 100)
x = 3 * np.exp(-2 * t)
y = 0.5 * np.exp(t)
X = np.stack((x, y), axis=-1)  # First column is x, second is y

sbr_optimizer = ps.optimizers.sbr.SBR(num_warmup=100,
                              num_samples=100,
                              mcmc_kwargs={"seed": 44,
                                           "num_chains": 2})

dif = ps.SINDyDerivative(kind='kalman', alpha = 0.01)

feature_library = ps.PolynomialLibrary(degree=2, include_bias=True)
model = ps.SINDy(optimizer=sbr_optimizer, feature_library=feature_library, differentiation_method=dif)
model.fit(X, t=t)
model.print() # Print the model

Error:

{
    "name": "AttributeError",
    "message": "'tuple' object has no attribute 'reshape'",
    "stack": "---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[42], line 20
     18 feature_library = ps.PolynomialLibrary(degree=2, include_bias=True)
     19 model = ps.SINDy(optimizer=sbr_optimizer, feature_library=feature_library, differentiation_method=dif)
---> 20 model.fit(X, t=t)
     21 model.print() # Print the model

File ~/micromamba/envs/VenturelliLab/lib/python3.10/site-packages/pysindy/pysindy.py:407, in fit(self, x, t, x_dot, u, multiple_trajectories, unbias, quiet, ensemble, library_ensemble, replace, n_candidates_to_drop, n_subset, n_models, ensemble_aggregator)
    385 def score(self, x, t=None, x_dot=None, u=None, metric=r2_score, **metric_kws):
    386     \"\"\"
    387     Returns a score for the time derivative prediction produced by the model.
    388 
    389     Parameters
    390     ----------
    391     x: array-like or list of array-like, shape (n_samples, n_input_features)
    392         Samples from which to make predictions.
    393 
    394     t: float, numpy array of shape (n_samples,), or list of numpy arrays, optional \\
    395             (default None)
    396         Time step between samples or array of collection times. Optional,
    397         used to compute the time derivatives of the samples if x_dot is not
    398         provided.
    399         If None, the default time step ``t_default`` will be used.
    400 
    401     x_dot: array-like or list of array-like, shape (n_samples, n_input_features), \\
    402             optional (default None)
    403         Optional pre-computed derivatives of the samples. If provided,
    404         these values will be used to compute the score. If not provided,
    405         the time derivatives of the training data will be computed using
    406         the specified differentiation method.
--> 407 
    408     u: array-like or list of array-like, shape(n_samples, n_control_features), \\
    409             optional (default None)
    410         Control variables. If ``multiple_trajectories==True`` then u
    411         must be a list of control variable data from each trajectory.
    412         If the model was fit with control variables then u is not optional.
    413 
    414     metric: callable, optional
    415         Metric function with which to score the prediction. Default is the
    416         R^2 coefficient of determination.
    417         See `Scikit-learn \\
    418         <https://scikit-learn.org/stable/modules/model_evaluation.html>`_
    419         for more options.
    420 
    421     metric_kws: dict, optional
    422         Optional keyword arguments to pass to the metric function.
    423 
    424     Returns
    425     -------
    426     score: float
    427         Metric function value for the model prediction of x_dot.
    428     \"\"\"
    430     if t is None:
    431         t = self.t_default

File ~/micromamba/envs/VenturelliLab/lib/python3.10/site-packages/pysindy/utils/axes.py:848, in concat_sample_axis(x_list)
    846     new_axes = {\"ax_sample\": 0, \"ax_coord\": 1}
    847     n_samples = np.prod([x.shape[ax] for ax in sample_ax_inds])
--> 848     arr = AxesArray(x.reshape((n_samples, x.shape[x.ax_coord])), new_axes)
    849     new_arrs.append(arr)
    850 return np.concatenate(new_arrs, axis=new_arrs[0].ax_sample)

AttributeError: 'tuple' object has no attribute 'reshape'"
}

And there are problems by using the other optimizers: For example:

import numpy as np
import pysindy as ps

t = np.linspace(0, 1, 100)
x = 3 * np.exp(-2 * t)
y = 0.5 * np.exp(t)
X = np.stack((x, y), axis=-1)  # First column is x, second is y

ssr_optimizer = ps.optimizers.SSR()

dif = ps.SINDyDerivative(kind='kalman', alpha = 0.01)

feature_library = ps.PolynomialLibrary(degree=2, include_bias=True)
model = ps.SINDy(optimizer=ssr_optimizer, feature_library=feature_library, differentiation_method=dif)
model.fit(X, t=t)
model.print() # Print the model

Error:

{
    "name": "TypeError",
    "message": "BaseOptimizer.__init__() got an unexpected keyword argument 'fit_intercept'",
    "stack": "---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[49], line 10
      6 y = 0.5 * np.exp(t)
      7 X = np.stack((x, y), axis=-1)  # First column is x, second is y
---> 10 ssr_optimizer = ps.optimizers.SSR()
     13 dif = ps.SINDyDerivative(kind='kalman', alpha = 0.01)
     15 feature_library = ps.PolynomialLibrary(degree=2, include_bias=True)

File ~/micromamba/envs/VenturelliLab/lib/python3.10/site-packages/pysindy/optimizers/ssr.py:103, in __init__(self, alpha, max_iter, ridge_kw, normalize_columns, fit_intercept, copy_X, criteria, kappa, verbose)
     87 def __init__(
     88     self,
     89     alpha=0.05,
   (...)
     97     unbias=True,
     98 ):
     99     super(SSR, self).__init__(
    100         max_iter=max_iter,
    101         copy_X=copy_X,
    102         normalize_columns=normalize_columns,
--> 103         unbias=unbias,
    104     )
    106     if alpha < 0:
    107         raise ValueError(\"alpha cannot be negative\")

TypeError: BaseOptimizer.__init__() got an unexpected keyword argument 'fit_intercept'"
}

PySINDy/Python version information:

Version: PySINDy: 1.7.6.dev335+g024bc9e 3.10.14 Python: 3.10.14

Jacob-Stevens-Haas commented 4 weeks ago

Interesting. I was not able to reproduce, either on current master or on commit 024bc9e. The functions behaved like they should.

What was the output of the pip install?

ggmirandac commented 4 weeks ago

Yeah, sure, Here it is: I try the same pip command:

 pip3 install "pysindy[sbr]@git+https://github.com/dynamicslab/pysindy"

Then: This is the pip output:


Collecting pysindy@ git+https://github.com/dynamicslab/pysindy (from pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy)
  Cloning https://github.com/dynamicslab/pysindy to /private/var/folders/q9/tkrlgcq55bl3f5qwz6jzws_r0000gn/T/pip-install-4xl5chx0/pysindy_496e6a7664f74c239826b8ba12609d89
  Running command git clone --filter=blob:none --quiet https://github.com/dynamicslab/pysindy /private/var/folders/q9/tkrlgcq55bl3f5qwz6jzws_r0000gn/T/pip-install-4xl5chx0/pysindy_496e6a7664f74c239826b8ba12609d89
  Resolved https://github.com/dynamicslab/pysindy to commit 3503c158e4e8a89163c300e47f827ab4f45e53fd
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: scikit-learn!=1.5.0,>=1.1 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.4.1.post1)
Requirement already satisfied: derivative>=0.5.4 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.6.1)
Requirement already satisfied: numpyro in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.15.0)
Requirement already satisfied: jax in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.4.25)
Requirement already satisfied: arviz==0.17.1 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.17.1)
Requirement already satisfied: scipy<1.13.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.12.0)
Requirement already satisfied: setuptools>=60.0.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (69.2.0)
Requirement already satisfied: matplotlib>=3.5 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (3.8.3)
Requirement already satisfied: numpy<2.0,>=1.22.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.24.4)
Requirement already satisfied: packaging in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (24.0)
Requirement already satisfied: pandas>=1.4.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (2.2.1)
Requirement already satisfied: xarray>=0.21.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (2024.3.0)
Requirement already satisfied: h5netcdf>=1.0.2 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.3.0)
Requirement already satisfied: typing-extensions>=4.1.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (4.10.0)
Requirement already satisfied: xarray-einstats>=0.3 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.7.0)
Requirement already satisfied: joblib>=1.2.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from scikit-learn!=1.5.0,>=1.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from scikit-learn!=1.5.0,>=1.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (3.4.0)
Requirement already satisfied: ml-dtypes>=0.2.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from jax->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.3.2)
Requirement already satisfied: opt-einsum in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from jax->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (3.3.0)
Requirement already satisfied: jaxlib>=0.4.14 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from numpyro->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.4.23.dev20240224)
Requirement already satisfied: multipledispatch in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from numpyro->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.6.0)
Requirement already satisfied: tqdm in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from numpyro->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (4.66.2)
Requirement already satisfied: h5py in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from h5netcdf>=1.0.2->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (3.10.0)
Requirement already satisfied: contourpy>=1.0.1 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.2.0)
Requirement already satisfied: cycler>=0.10 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (4.50.0)
Requirement already satisfied: kiwisolver>=1.3.1 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.4.5)
Requirement already satisfied: pillow>=8 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (10.2.0)
Requirement already satisfied: pyparsing>=2.3.1 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (3.1.2)
Requirement already satisfied: python-dateutil>=2.7 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from matplotlib>=3.5->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (2.9.0)
Requirement already satisfied: pytz>=2020.1 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pandas>=1.4.0->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from pandas>=1.4.0->arviz==0.17.1->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (2024.1)
Requirement already satisfied: six in ./micromamba/envs/VenturelliLab/lib/python3.10/site-packages (from multipledispatch->numpyro->pysindy@ git+https://github.com/dynamicslab/pysindy->pysindy[sbr]@ git+https://github.com/dynamicslab/pysindy) (1.16.0)
Building wheels for collected packages: pysindy
  Building wheel for pysindy (pyproject.toml) ... done
  Created wheel for pysindy: filename=pysindy-1.7.6.dev346+g3503c15-py3-none-any.whl size=128987 sha256=ee995e8888ecf1c68326ee8522c6040091d2870c5f9e20b91f0b4dcf54db62e1
  Stored in directory: /private/var/folders/q9/tkrlgcq55bl3f5qwz6jzws_r0000gn/T/pip-ephem-wheel-cache-9zpie3td/wheels/67/70/e9/7fe05f22d0972893ddacb2ad333099204bdbefbab173594a2d
Successfully built pysindy
Installing collected packages: pysindy
Successfully installed pysindy-1.7.6.dev346+g3503c15
ggmirandac commented 4 weeks ago

Hi,

I manage to solve it somehow. The main issue is that the package cvxpy was not installed in the dependencies of the package, so I had to manually installed it.

Bests,