search

dynamicslab / pysindy

A package for the sparse identification of nonlinear dynamical systems from data
https://pysindy.readthedocs.io/en/latest/
Other
1.36k stars 304 forks source link

Make models guaranteed pickleable #456

Closed Jacob-Stevens-Haas closed 5 months ago

Jacob-Stevens-Haas commented 5 months ago

Is your feature request related to a problem? Please describe.

Currently pickleing or unpickling (even using dill) can fail on MISOR objects. When unpickling

RecursionError: maximum recursion depth exceeded while calling a Python object

or a an error about being unable to pickle PyCapsule objects when pickling.

Describe the solution you'd like

Serializating objects correctly is one of those hard problems that are solveable, but probably beyond my ken (and probably also relies on gurobipy's backwards compatability)

Describe alternatives you've considered

  1. a function strip_optimizer(self) that returns a new, fitted model instance with no optimizer and pinned coefs solves the current issue, but there's no guarantee that feature libraries or derivative methods in the future are pickleable. score and predict don't utilize the optimizer, so this is meets my use case, but doesn't help if someone wants to unpickle a model and post-mortem the optimization step.
  2. Overload __reduce__ in that MIOSR to prevent even attempting to pickle
  3. Overload __reduce__ in that MIOSR to instead pickle a similar object that saves useful post-mortem information from the gurobi objects, but which is non-functional as an optimizer after pickling.
  4. Remove self.model from MIOSR objects' instance dictionaries.

Additional context

If we ever tried to save the cvxpy problems associated with an SR3 optimizer, we may end up with the same issue (depending on how well pickling is supported by that package)

adding @wesg52 in case you have any experience with pickling gurobi objects.