search

openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

[Feature Request] Support a list of frameworks in runbenchmark.py args #483

Closed Innixma closed 1 year ago

Innixma commented 1 year ago

Currently to run 3 frameworks h2oautoml, autogluon, autogluon:latest:

python3 runbenchmark.py h2oautoml validation 1h4c -m aws
python3 runbenchmark.py autogluon validation 1h4c -m aws
python3 runbenchmark.py autogluon:latest validation 1h4c -m aws

Ideally:

python3 runbenchmark.py [h2oautoml, autogluon, autogluon:latest] validation 1h4c -m aws

Benefits of the new logic is that compute can be used when using multiple instances more effectively (for example, running instances on both h2o and autogluon at the same time within the same runbenchmark.py call.

PGijsbers commented 1 year ago

Duplicate of https://github.com/openml/automlbenchmark/issues/169