search

openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

Benchmark name "amlb.yaml" causes error. #591

Closed sedol1339 closed 9 months ago

sedol1339 commented 9 months ago

Hello! I did not find a full list of 104 AMLB datasets in the repo (where are they?). When I create amlb.yaml manually and fill it with 104 datasets, runbenchmark.py yields error:

No task available.

When I rename amlb.yaml to any other name, it works. Pretty strange behaviour.

PGijsbers commented 9 months ago

The datasets are not stored in the repository. If you can provide the exact commands you executed I might be able to help you better.

In general, the datasets used in our paper are defined on openml with task collections. That is, if you want to run our classification problems, it would look like python runbenchmark.py FRAMEWORK openml/s/271 ... and for regression it is python runbenchmark.py FRAMEWORK openml/s/269 ....

sedol1339 commented 9 months ago

Thank you