Improve documentation for the command line

[oeway]

I am trying to run model testing from a doi.

In the docs it said I should do bioimageio test-model -m <MODEL>

But it said -m is not an option:

(bioimageio.core) weiouyang@europa:~$ bioimageio test-model -m 10.5281/zenodo.6224243
site.config.json: 5.69kiB [00:00, 2.26MiB/s]                                             
...
bioimageio.spec 0.4.5post2
implementing:
        collection RDF 0.2.2
        general RDF 0.2.2
        model RDF 0.4.5
bioimageio.core 0.5.1
Usage: bioimageio test-model [OPTIONS] MODEL_RDF
Try 'bioimageio test-model -h' for help.

Error: No such option: -m

Remove the -m option works though.

For future improvements, from the command line, it also prints:

/home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/raw.githubusercontent.com/bioimage-io/bioimage.io/main/site.config.json. Skipping download of https://raw.githubusercontent.com/bioimage-io/bioimage.io/main/site.config.json.
  warnings.warn(f"found cached {local_path}. Skipping download of {uri}.")
/home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/bioimage-io.github.io/collection-bioimage-io/collection.json. Skipping download of https://bioimage-io.github.io/collection-bioimage-io/collection.json.
  warnings.warn(f"found cached {local_path}. Skipping download of {uri}.")

A full log looks like this:

Click to expand!

(bioimageio.core) weiouyang@europa:~$ bioimageio test-model 10.5281/zenodo.6224243 --devices cpu /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/raw.githubusercontent.com/bioimage-io/bioimage.io/main/site.config.json. Skipping download of https://raw.githubusercontent.com/bioimage-io/bioimage.io/main/site.config.json. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/bioimage-io.github.io/collection-bioimage-io/collection.json. Skipping download of https://bioimage-io.github.io/collection-bioimage-io/collection.json. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") bioimageio.spec 0.4.5post2 implementing: collection RDF 0.2.2 general RDF 0.2.2 model RDF 0.4.5 bioimageio.core 0.5.1 /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/rdf.yaml. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/rdf.yaml. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/schema.py:46: ValidationWarning: tags: Missing tags for categories: [{'modality': ['electron-microscopy', 'cryo-electron-microscopy', 'fluorescence-light-microscopy', 'transmission-light-microscopy', 'super-resolution-microscopy', 'x-ray-microscopy', 'force-microscopy', 'high-content-imaging', 'whole-slide-imaging', 'other']}, {'dims': ['2d', '3d', '2d-t', '3d-t']}, {'content': ['cells', 'nuclei', 'extracellular-vesicles', 'tissue', 'plant', 'mitochondria', 'vasculature', 'cell-membrane', 'brain', 'whole-organism']}, {'framework': ['tensorflow', 'pytorch', 'tensorflow.js']}, {'software': ['ilastik', 'imagej', 'fiji', 'imjoy', 'deepimagej', 'n2v']}, {'network': ['unet', 'densenet', 'resnet', 'inception', 'shufflenet']}, {'task': ['semantic-segmentation', 'instance-segmentation', 'object-detection', 'image-classification', 'denoising', 'image-restoration', 'image-reconstruction', 'in-silico-labeling']}] warnings.warn(msg, category=ValidationWarning) /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/schema.py:46: ValidationWarning: weights:torchscript: missing 'pytorch_version' warnings.warn(msg, category=ValidationWarning) /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/fixed_zero_mean_unit_variance.ijm. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/fixed_zero_mean_unit_variance.ijm. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/hpaseg-cover.png. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/hpaseg-cover.png. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/README.md. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/README.md. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/sample_input_0.tif. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/sample_input_0.tif. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/sample_output_0.tif. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/sample_output_0.tif. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/test_nuclei_input.npy. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/test_nuclei_input.npy. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/test_nuclei_output.npy. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/test_nuclei_output.npy. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") /home/weiouyang/miniconda3/envs/bioimageio.core/lib/python3.8/site-packages/bioimageio/spec/shared/_resolve_source.py:349: UserWarning: found cached /tmp/bioimageio_cache/https/zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/nuclei_model.pt. Skipping download of https://zenodo.org/api/files/9f580ca7-5f85-41ac-be2b-201172ed09b3/nuclei_model.pt. warnings.warn(f"found cached {local_path}. Skipping download of {uri}.") code/__torch__/torch/nn/functional.py:21: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument. Model test for 10.5281/zenodo.6224243 using torchscript weight format has passed.

While the outputs looks long, the line really matters is the last line said the test has passed.

Would it be possible to make these warning messages more cleaner? (I also see the message when running bioimageio test-model -h)

[constantinpape]

The docs are outdated. I will update them. (Done.)

[constantinpape]

For the rest: I agree, I have complained with @FynnBe about all these warnings already but there is some reason for it...

[FynnBe]

Yes the cache warnings are annoying, but I am afraid of just removing them without updating the cache mechanism. we currently simply go by url, which is very dangerous; thus the warnings. (see https://github.com/bioimage-io/spec-bioimage-io/issues/289)