hungyiwu commented 4 years ago

Expected behavior

segmentify should work for non-square images (width != length, ex. 512x512 is square, 511x512 is not square)

Potentially relevant issues/PR

11 #13

How to reproduce the bug

Below is a slightly modified version of https://github.com/transformify-plugins/segmentify/blob/master/examples/launch.py

Run the code as is, popping the napari window; click execute button, confirm working
Un-comment the two lines right before napari.gui_qt; run the code, popping the napari window, click execute button, and confirm it crashes
```
import os
import napari
from segmentify import segmentation
import numpy as np
from skimage import io
```

parse input file

example_image = os.path.join(os.path.abspath(os.path.dirname(file)), "hpa.png") example_labels = os.path.join(os.path.abspath(os.path.dirname(file)), "hpa_labels.tif")

example_image = io.imread(example_image) example_labels = io.imread(example_labels)

WARNING: uncomment these two lines below to reproduce bug

example_image = example_image[:, 1:]

example_labels = example_labels[:, 1:]

##########################################################

with napari.gui_qt(): viewer = napari.Viewer()

# instantiate the widget
gui = segmentation.Gui()

# add our new widget to the napari viewer
viewer.window.add_dock_widget(gui)

# keep the dropdown menus in the gui in sync with the layer model
viewer.layers.events.changed.connect(lambda x: gui.refresh_choices())

gui.refresh_choices()

# load data
viewer.add_image(example_image)
viewer.add_labels(example_labels)


## Full error message

Segmenting... WARNING: Traceback (most recent call last): File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/magicgui-0.1.5-py3.8.egg/magicgui/core.py", line 199, in self.call_button.clicked.connect(lambda checked: self.call()) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/magicgui-0.1.5-py3.8.egg/magicgui/core.py", line 526, in call value = self.func(_kwargs) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/gui.py", line 29, in segmentation clf, features = fit(data, initial_labels.data, featurizer=featurizer.value) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/semantic/main.py", line 129, in fit padded_features = unet_featurize(padded_image, featurizer) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/semantic/main.py", line 59, in unet_featurize features = model(image) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/torch-1.5.1-py3.8-linux-x86_64.egg/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, *kwargs) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/model/unet.py", line 57, in forward x = self.decode_4(x, skip_4) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/torch-1.5.1-py3.8-linux-x86_64.egg/torch/nn/modules/module.py", line 550, in call result = self.forward(input, kwargs) File "/home/hungyiwu/.venv/segmentify/lib/python3.8/site-packages/segmentify-0.1.1-py3.8.egg/segmentify/model/layers/unet_layers.py", line 63, in forward output += skip RuntimeError: The size of tensor a (62) must match the size of tensor b (63) at non-singleton dimension 3 Aborted (core dumped)



## Environment, version, and other information
python 3.8.2
napari 0.3.5
napari-plugin-engine 0.1.6
napari-svg 0.1.3
segmentify 0.1.1 (installed by cloning this repo and `python setup.py install` the `master` branch)
Ubuntu 20.04
python venv
`pip freeze > `[requirements.txt](https://github.com/transformify-plugins/segmentify/files/4837652/requirements.txt)

sofroniewn commented 4 years ago

segmentify should work for non-square images (width != length, ex. 512x512 is square, 511x512 is not square)

If you wanted to work on a fix for this @hungyiwu I'd be happy to review and merge! It likely involves some of the padding code

hungyiwu commented 4 years ago

I looked into the models and the issue is not strictly square vs. non-square images, but images of shape 512x512 vs. otherwise, as this following didn't work either

example_image = example_image[1:, 1:]
example_labels = example_labels[1:, 1:]

I realized the original model was trained using 512x512 images and likely only support predictions of this input shape. So perhaps not really a bug but we can add documentation to remind users this pre-trained model supports this shape, and if they want something else they can train their own model for that.

Any thoughts? @sofroniewn