search

ODINN-SciML / MassBalanceMachine

Global machine learning glacier mass balance model, capable of assimilating all sources of glaciological and remote sensing data
MIT License
15 stars 11 forks source link

Fix pickling problem with closure in custom loss by using dill #36

Open khsjursen opened 3 months ago

khsjursen commented 3 months ago

When saving a grid search object with the custom XGBRegressor class and objective function, the following does not work: joblib.dump(gs_obj, dir + '/' + 'custom_loss_cv_grid.pkl')

The custom objective function cannot be pickled since it is defined within a closure. Using dill instead of pickle solves the problem:

import dill as pickle

# Run grid search
gs_obj = GridSearchCV(..)

# Save grid search object
with open(dir + '/' + 'custom_loss_cv_grid.pkl', 'wb') as f:
    pickle.dump(gs_obj, f)

Install with: pip install dill

JulianBiesheuvel commented 2 months ago

Currently, this is implemented in the CustomXGBoostRegressor class directly. In addition, to loading a pre-existing model/gridsearch. Eventually, when more models are added this feature should be part of a wrapper class (or something similar).