search

openml / openml-r

R package to interface with OpenML
http://openml.github.io/openml-r/
Other
96 stars 37 forks source link

upload the learner object with each run #411

Open giuseppec opened 6 years ago

giuseppec commented 6 years ago

There are sometimes bugs in mlr (e.g. https://github.com/mlr-org/mlr/issues/2218) or OpenML (e.g. 171aee917553e072a61de6d52fa0a7a1a4f6e715 ) causing critical issues (e.g. some hyperparameters are not uploaded properly on the OpenML server). Maybe we should always upload the learner object just to be sure that if something went wrong, one can still get the hyperparameters from the uploaded mlr learner object. I thought of uploading just the result of getHyperPars(lrn) instead of the learner object itself, but this still does not solve the issue when there are also bugs in mlr (e.g. https://github.com/mlr-org/mlr/issues/2218).

joaquinvanschoren commented 6 years ago

+1

On Mon, 12 Mar 2018 at 17:58 Giuseppe Casalicchio notifications@github.com wrote:

There are sometimes bugs in mlr or OpenML causing an issue that hyperparameters are not stored properly on the OpenML server. Maybe we should always upload the learner object just to be sure that if something went wrong, one can still get the hyperparameters from the uploaded mlr learner object. I also thought of uploading just the result of getHyperPars(lrn) instead of the learner object itself, but this still does not solve the issue when there are also bugs in mlr (e.g. mlr-org/mlr#2218 https://github.com/mlr-org/mlr/issues/2218).

Uploading the learner also solves the issue for learners where user-defined functions are passed as hyperparameters.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openml/openml-r/issues/411, or mute the thread https://github.com/notifications/unsubscribe-auth/ABpQV9sVt60-QF7WmykYoW8RTW4Vb-x0ks5tdqlBgaJpZM4SnA7r .

-- Thank you, Joaquin

giuseppec commented 6 years ago

@joaquinvanschoren do you think this could be a file storage problem for the server? A mlr learner object can be up to 0.1 MegaByte and we currently have 9.000.000 runs now. Or are these peanuts?