Writing results back to Tiled from the segmentation model.

[dylanmcreynolds]

We want to enhance the code that runs the segmentation model to write results and status back to Tiled.

What repo will this work be done in?

[TibbersHao]

Getting back to saving structures in tiled for seg results, this is what we have decided:

├── mlex_store
│   └── rec20190524_085542_clay_testZMQ_8bit
│       ├── masks
│       │   └── mask_array_container
│       └── results
│           └── uid0001
│               └── seg_result_array_container
└── reconstruction
    └── rec20190524_085542_clay_testZMQ_8bit
        └── 20190524_085542_clay_testZMQ_

And the seg result array saved for each instance will have metadata that points back to the corresponding masks and images used for training:

metadata={
        'recon_uri': tiled_dataset.recon_client.uri,
        'mask_uri': tiled_dataset.mask_client.uri, 
        'model': model, 
        }

Regarding the loss and model saving, we will have more insights with Xiaoya's work which will require another discussion next week.

[dylanmcreynolds]

Thanks for adding the note. I think this is a great start and that when we integrate this into the system, we may find that we need to tweak the structure a little.

[TibbersHao]

New topics to be discussed:

Better way to handle large arrays as the np.concat will likely be a bad idea for GB-sized data.

Potential ideas to work around this:

• Dask
• Save results of individual frames on-the-fly, and handle the stacking/concatenation in Tiled.

[TibbersHao]

Update: we will need a discussion for this before making further adjustments.

[TibbersHao]

close with #12