search

FG-AI4H-TG-Symptom / fgai4h-tg-symptom-assessment-mmvb-backend

A backend of a minimal minimal viable benchmark designed and developed by the members of the sub-group (topic group) “Symptom assessment”, which is the part of the ITU focus group “Artificial Intelligence for Health”
6 stars 4 forks source link

Berlin model compatible benchmark runner #69

Open neopostmodern opened 4 years ago

maccinza commented 4 years ago

Where can I find a minimal berlin model schema so we can start the work for adapting benchmark runner? Similarly to the toy ais, would we like to keep some flexibility to work with both london and berlin models or the goal is to make it fully compatible with berlin model only?

neopostmodern commented 4 years ago

see https://github.com/FG-AI4H-TG-Symptom/fgai4h-tg-symptom-assessment-mmvb-backend/issues/66#issuecomment-633551540

the benchmark runner in an ideal world is agnostic of the model i think, it just sends out a JSON blob and checks the answers against the answer-schema