bioimage-io / collection-bioimage-io

(deprecated in favor of bioimage-io/collection) RDF collection for BioImage.IO
5 stars 9 forks source link

Update EMDiffuse-n #705

Open bioimageiobot opened 5 months ago

bioimageiobot commented 5 months ago

This is an automatic PR created by the @bioimageiobot regarding changes to the resource item 10.5281/zenodo.10575472. The following version(s) will be added:

Please review the changes and make sure the new item or version(s) pass the following check list:

Maintainers: @luchixiang

Note: If you updated or re-uploaded another version for the current item on Zenodo, this PR won't be changed automatically. To proceed, you can do the following:

  1. Block this version, but keep looking for future versions: Edit the current resource.yaml and keep the top-level status field as accepted, but change the status under the current version to blocked.
  2. Accept this version and keep looking for future versions: Merge this PR for now.
  3. Keep proposed version(s) (and this resource in general if it is new) as pending: Close this PR without merging.

    Then wait for the CI on the main branch to complete. It should detect the new version(s) and create another PR for the new version(s).

Previous PRs of this resource: none

bioimageiobot commented 5 months ago

preview

Luchixiang commented 5 months ago

Hi! How do I check whether the model is uploaded successfully or not? And can I modify the model name or tags or descriptions after uploading?

Thanks

esgomezm commented 5 months ago

Hi @Luchixiang Thank you for uploading the model. Before the model gets uploaded, it needs to pass different tests. You can see them below your comment (I add a screenshot). You can see that none of the two models passed the validation tests (dynamic validation), i.e., we cannot load your model and run it on the provided test images. This usually happens when something is missing in the model rdf.yaml file or the pytorch model is not exported properly. Here are some comments:

Luchixiang commented 5 months ago

Hi! Does the model output of test_input.npy needs to match test_output.npy in order to pass validation? The diffusion model is a generative model, so its output should be slightly different each time.

esgomezm commented 5 months ago

Sorry, yes, the CI we use to test the models assumes that the output is always the same. This is something we are working on. Unfortunately, we do not have a solution for it yet. Sometimes, when fixing the seed of the diffusion model, it is possible to get the same output. Could you check if you could do so? Also, it would be very helpful for us if you could get the model running properly and just the validation saying that the output is not the same (similar to this case https://github.com/bioimage-io/collection-bioimage-io/pull/686). This way, we know that we can at least, load and run your model, and we could develop further pipelines to test generative approaches.