search

rademacher-p / stats-learn

Python package for statistical learning with a Bayesian focus
https://rademacher-p.github.io/stats-learn/
MIT License
1 stars 0 forks source link

Fix treatment of `predictor.set_params` in `results` utils #42

Open rademacher-p opened 7 months ago

rademacher-p commented 7 months ago

result utils allow specification of different parameter sets that the predictors are evaluated for. These utils assume that the predictors can set their parameters without necessitating a reset of their learned params. This is not generally the case.

The solution is to rework data_assess and model_assess to force a refit based on some new predictor attribute that indicates "in-place" parameters. At least, the utils can raise an exception if the params argument lists parameters that cannot be modified without refit.

rademacher-p commented 7 months ago

Special cases where reset is not needed include fixed models and Bayesian models with special parameters. General Bayesian models, NN predictors, etc., require a refit on parameter set.