search

automl / HPOBench

Collection of hyperparameter optimization benchmark problems
Apache License 2.0
136 stars 35 forks source link

[WIP] Add Federated HPO benchmark #156

Open rayrayraykk opened 2 years ago

rayrayraykk commented 2 years ago

Hello there! I am from FederatedScope team. Considering some uniqueness of HPO for FL algorithms (FedHPO), we build a benchmark suite for federated HPO: FedHPO-Bench, and our paper "FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization" can be found here.

We think that HPOBench is a very popular and influential library, so we try to expose our FedHPO benchmarks through the interfaces of HPOBench. In this PR, I implement a tabular benchmark with FedAvg+CNN on FEMNIST as a first try, which would not introduce any additional dependency to your library. And we will try to contribute more FedHPO benchmarks in the future!

PhMueller commented 2 years ago

Hi,

Thank you very much for reaching out to us. We are always looking forward to extending the HPOBench with exciting new benchmarks. Unfortunately, I am on vacation for the next two weeks.

I will reach out to you as soon as I am back.