comic / evalutils

evalutils helps users create extensions for grand-challenge.org
https://grand-challenge.org
MIT License
23 stars 9 forks source link

Parameter specification not working for `evalutils init algorithm` #318

Closed joeranbosma closed 2 years ago

joeranbosma commented 2 years ago

Description

The questionnaire when initializing an algorithm with evalutils is broken. Instead of asking for the required inputs, it throws an error. This happens for the number of op CPUs and GPUs. I have tested this on both macOS and WSL2 on Windows 11, and the error messages were exactly the same.

If possible, it would be amazing if this issue would be fixed before the Algorithm workshop on 19 October 2021.

What I Did

evalutils init algorithm VesselSegmentationContainer
> Segmentation (but same error persists for Detection/Classification)

image

This issue can be circumvented by adding the flag manually. Doing so yields a similar issue with the number of GPUs: image

Also, shouldn’t it be possible to have an algorithm that does not require a GPU? Adding that flag as well results in a different order of questions, but it succeeds in the end: image

jmsmkn commented 2 years ago

Most of (if not all) these options were removed in https://github.com/comic/evalutils/commit/fd791e0f1715d78b3766ac613371c447607e411d - can you check to see if this is an issue on master?

joeranbosma commented 2 years ago

Hi @jmsmkn, installing from source (which was version 0.3.0) did indeed solve the issues!

Would it be possible to create a new release for this version such that the participants of the workshop can just use pip install evalutils?

jmsmkn commented 2 years ago

Version 0.3.0 released https://pypi.org/project/evalutils/