Requirements for BraTS Challenge

[aristizabal95]

This is an issue to keep track of changes and requirements that must be met in order to make Medperf part of the BraTS Challenge.

Competitors (Model Owners) must be able to:

• [x] have/create their own account. (CLI command? or created beforehand by the MedPerf Admin?)
• [x] log into their accounts.
• [x] mlcommons/medperf#58
• [x] mlcommons/medperf#57
• [x] Register new cubes.
• [x] Request cube association to benchmarks.

Data Owners must be able to:

• [x] have/create their own account.
• [x] log into their accounts.
• [x] Prepare dataset for a benchmark.
• [x] Register a prepared dataset to the server.
• [x] Request dataset association to benchmarks.
• [x] Run a benchmark execution flow.
• [x] Upload execution results to the server.

Benchmark Owner must be able to:

• [x] Create new benchmarks.
• [x] Approve dataset association requests.
• [x] Approve model association requests.
• [x] mlcommons/medical-internal#43

Medperf Admin must be able to (Do they require the CLI?):

• [x] Assign user role (More specifically, determine who is a benchmark owner).

Additional requirements:

• [ ] Extensive wiki, including diagrams, tutorials, full demo, and explanation of each section of the code.

Is there anything else that must be implemented?

[aristizabal95]

Medperf admin creates accounts, at least for the early stage.

[aristizabal95]

Instructions for each agent on the system (Data owner, model owner, benchmark owner). Create README for each case to show how to use the platform

[hasan7n]

Closing as outdated