search

openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
391 stars 130 forks source link

Test PR of using custom config path instead of editing original configs #501

Closed Innixma closed 1 year ago

Innixma commented 1 year ago

Test PR of using custom config path instead of editing original configs

Success:

python ../automlbenchmark/runbenchmark.py AutoGluon_zs_XT:zeroshot test test -u ../automlbenchmark/custom_configs

Failure:

python ../automlbenchmark/runbenchmark.py AutoGluon_zs_XT:zeroshot test test -u ../automlbenchmark/custom_configs -m aws -p 10

Error:

[INFO] [amlb:20:58:35.801] Running benchmark `AutoGluon_zs_XT:zeroshot` on `test` framework in `local` mode.
[INFO] [amlb.frameworks.definitions:20:58:35.838] Loading frameworks definitions from ['/repo/resources/frameworks.yaml'].
[ERROR] [amlb:20:58:35.895] 
ERROR:
Incorrect tag `zeroshot`: only those among ['stable', 'latest', '2020Q2', '2021Q3'] are allowed.
Innixma commented 1 year ago

Apologies, meant to open within my own fork. Closing.