mlcommons / medperf

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
https://www.medperf.org
Apache License 2.0
146 stars 32 forks source link

Pass expected institutions during benchmark creation #624

Open aristizabal95 opened 1 month ago

aristizabal95 commented 1 month ago

This PR allows benchmark owners to optionally specify which sites are expected to participate on a benchmark. This is done through a CSV file that contains two columns: email and institution. This additiona provides the following benefits:

The institutions list are stored as a dictionary in the benchmark object. The CSV file is parsed by the medperf CLI to extract the information in the expected format. This list can only be visible by the benchmark owner.

github-actions[bot] commented 1 month ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅