openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

Add computer readable list with framework meta-data #164

Open PGijsbers opened 3 years ago

PGijsbers commented 3 years ago

Useful meta-data to collect:

It allows one to automatically select all frameworks compatible with a benchmark, which also makes it easier/clearer to support frameworks which do not implement all data/task types. Additionally we can make integration of the automl system with the benchmark framework easier if we can automatically determine whether or not to do missing value imputation or other preprocessing.

PGijsbers commented 3 years ago

The only sensible place (to me) to add this seems the frameworks.yaml, as the same framework may have different levels of support in different versions.