has2k1 / scikit-misc

Miscellaneous tools for data analysis and scientific computing
https://has2k1.github.io/scikit-misc/stable
BSD 3-Clause "New" or "Revised" License
37 stars 9 forks source link

unable to save the model with pickle #26

Closed Julien-Yacine closed 1 year ago

Julien-Yacine commented 1 year ago

Hi,

I tried to save the model using pickle and I get the error TypeError: self._base cannot be converted to a Python object for pickling.

from skmisc.loess import loess
import numpy as np
import pickle

span_val = 0.1

x = np.random.random_sample((1000,))
y = np.random.random_sample((1000,))
model = loess(x,
             y, 
             span=span_val,                 
             normalize=False,
             surface="direct",
             family="gaussian",
)
model.fit()

pickle.dumps(model)

I succeded using skops but once I load the model I get Error: Segmentation fault (core dumped)

from skmisc.loess import loess
import numpy as np
import skops.io as sio

span_val = 0.1

x = np.random.random_sample((1000,))
y = np.random.random_sample((1000,))

model = loess(x,
             y, 
             span=span_val,                 
             normalize=False,
             surface="direct",
             family="gaussian",
)
model.fit()

sio.dump(model, "./model.skops", compression=0)

unknown_types = sio.get_untrusted_types(file="model.skops")

model_loaded = sio.load("./model.skops", trusted=unknown_types)
model_loaded.predict(x)

I use python 3.10.9 on Ubuntu 20.04.

Thank you for your help and for the library.

has2k1 commented 1 year ago

I've added pickling support. But, fitting a loess regression generates large & complex state to enable predictions and it would be cumbersome to save that state.

Pickling a loess object saves the initial parameters and the state recreated (running fit again) automatically when the model is loaded/unplickled. That means pickling a fitted model will not save compute time!