Modelzoo exporter command

frauzufall commented 4 years ago

We need a command & API to export trained models into modelzoo format.

Use cases

Export SavedModelBundle method

input: SavedModelBundle directory (including variables, saved_model.pb)
input: additional needed parameters (inputs, outputs, data type etc.)
Add missing information to existing YAML specification
input: A ZIP file with a SavedModelBundle and a model.yaml file

Goal

The exporter should be able to write the YAML specification and ask the user for missing parameters (like author names, a doi to the training data etc.).

Existing code

Writing the model is already implemented in Noise2Void in Fiji and should move from there to this repository, then N2V and others can use imagej-modelzoo as a library: https://github.com/juglab/N2V_fiji/blob/master/src/main/java/de/csbdresden/n2v/train/ModelSpecification.java

Tagging @esgomezm @ctr26 in case it might be possible to reuse code from DeepImageJ for the UI part (discussion started in this issue). @tibuch

esgomezm commented 4 years ago

I think @oeway and @FynnBe can help with this.

We were working in the update to YAML while N2V was being published so now we need to merge all the specifications. Does it make sense to create this file from Java or is it better to create it from Python (DeepImageJ only works with TF models)?

What about parsing an official template located in imagej-modelzoo?

tibuch commented 4 years ago

We are already working closely together with @oeway and @FynnBe regarding the yaml-specification. But this project here should be a consumer of the modelzoo and is purely on the Java side.

The published N2V uses the modelzoo format and is available here as a demo. As you can see in the n2v-sem-demo.model.yaml not all fields are filled out. Especially a link to the used training data is missing. Also author and documentation is currently added by hand editing the generated yaml file.

What do you think about joining forces and combine our efforts in this repo?

esgomezm commented 4 years ago

What do you think about joining forces and combine our efforts in this repo?

Cool idea!

esgomezm commented 4 years ago

How do you want to proceed?

This is just an idea:

Create a new general plugin for the imagej-modelzoo, let's say "modelzoo importer"
Through a GUI similar to DeepImageJ's one, this plugin parses the YAML file specified here and completes it with all the required information.
Should we try to define the YAML here here, according to the description given in bioimage-io/configuration? I'd say yes.

The main difference between current n2v-sem-demo and the version DeepImageJ needs is the integration of macros for the preprocessing and post-processing. What do you think is the best way to integrate them?

tibuch commented 4 years ago

I would suggest that we setup a call within the next two weeks where we can synch our two projects. After this we should be able to create a list of github issues which we can assign and implement.

I think the three points you mentioned are part of the required features and should all live in this repo.

The main difference between current n2v-sem-demo and the version DeepImageJ needs is the integration of macros for the preprocessing and post-processing. What do you think is the best way to integrate them?

In my opinion macro-integration is another layer on top of this layer.

esgomezm commented 4 years ago

I would suggest that we setup a call within the next two weeks where we can synch our two projects. After this we should be able to create a list of github issues which we can assign and implement.

Perfect. How do you want to proceed?

oeway commented 4 years ago

As discussed with @esgomezm and @dasv74 during NEUBIAS, we think it would be a nice addition to provide a web based BioEngine app for making model packages directly on bioimage.io.

We can basically generate fields from a schema file for the model config, we have a little experiment here.

The generated web interface will have all the parameters included in the yaml, they will be generated from the schema, so we make sure the generated yaml will be compatible with the specification.

Not only the yaml, but we can also add files and build the zip file within the bioengine app.

Further more, when the zip file with weights and yaml are made, the bioengine app can spin up a remote server and run a test with the model directly.

Of course, in the meantime, you can implement the same in Java natively. For longer term, a more centralized web based bioengine app would require less maintenance efforts, especially when we change the specification.

imagesc-bot commented 3 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/compatibility-of-imagej-tensorflow-with-imagej1/41295/6

imagej / imagej-modelzoo