openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

Fix errors for the remainder of `amlb` module. #575

Closed PGijsbers closed 4 months ago

PGijsbers commented 11 months ago

Opted out of one exception: there is a special case for the recovery script where run_mode is "script". This makes some fields optional, which then introduces some ~10 errors because in the normal workflow they are not optional and this is assumed in the code. I think it is probably possibly to refactor the recovery benchmark mode into something that is not actually a Benchmark as it doesn't really seem to use the internals that much in fetch mode. For reconnect it later acts as script, so there is probably some room to simplify the workflow and address the underlying issue.