mlcommons / medperf

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
https://www.medperf.org
Apache License 2.0
146 stars 32 forks source link

[FEATURE] log inference times #491

Open neuronflow opened 1 year ago

neuronflow commented 1 year ago

Is your feature request related to a problem? Please describe. We need to know the inference times of algorithms per exam.

Describe the solution you'd like Medperf should log and return inference times in milliseconds.

Additional context Email conversation with Hasan and Verena. Miccai challenge guidelines (attached) BIAS_ReportingGuideline.pdf

neuronflow commented 1 year ago

PS: ideally you can separate between timings:

Probably I missed some relevant points.