search

openml / automlbenchmark

OpenML AutoML Benchmarking Framework
https://openml.github.io/automlbenchmark
MIT License
399 stars 132 forks source link

FEDOT integration #563

Closed nicl-nno closed 1 year ago

nicl-nno commented 1 year ago

Hi!

I've added integration with FEDOT AutoML framework (https://github.com/aimclub/FEDOT) for tabular tasks. The local runs were evaluated successfully.

We also plan to add integration for time series forecasting in the next PR.

nicl-nno commented 1 year ago

@PGijsbers I added the changes according to the remaining comments. Also, I rebased my branch to main.

PGijsbers commented 1 year ago

Thanks! I’ll have a look later this week

PGijsbers commented 1 year ago

That seems to do the trick. Thanks for your effort! 🎉 As an aside, I did notice some problems running the validation benchmark (python runbenchmark.py fedot validation). For APSFailure there was a memory error (exit status 137), and for census-income it exceeded time limit. I am not sure how reproducible they are, I had a non-ideal setup (docker mode with QEMU). I don't want this to block the merge because 1) I don't know how reproducible this is and 2) it is likely not an integration error, but I did want to bring this to your attention. If you can reproduce this it is worth a look.

nicl-nno commented 1 year ago

Thank you, we will investigate it and try to reproduce.