search

Innixma / autogluon-benchmark

AutoGluon Benchmarking Repository
Apache License 2.0
1 stars 8 forks source link

User guide? #6

Open willsmithorg opened 2 years ago

willsmithorg commented 2 years ago

I would like to make trial changes to Autogluon and run A/B tests on the 50? datasets in the AutoGluon benchmarking set (kaggle + automl).

Could README.md be updated a little to explain how to do this, starting with a clean AMI (and ideally if you could recommend which AMI to use). I understand that m5.2xlarge is the preferred machine type to remain consistent with the original AutoML Benchmark paper.

Innixma commented 2 years ago

I updated the READMEs to give some more information, do note that the benchmark is quite involved due to its scale (isn't feasible to run on a single machine). If you have a proposed improvement to AutoGluon you'd like tested, you can open a PR with the changes and I can run the benchmark for you. The benchmark is 104 datasets (no kaggle atm).

We hope to provide a more approachable benchmark users can try out in the next couple months.