Docker image failing at Ilastik step

[eleonore-schneeg]

Hi,

I've been trying to run Steinbock from the command line. Everything works perfectly except for the Ilastik step, which can only run on the app itself. Basically this step: "steinbock classify ilastik run" fails

I've tried linux/Mac and windows. and multiple versions 16.1 and 15.0

Here is the error:

steinbock classify ilastik run

/opt/ilastik/ilastik-meta/lazyflow/lazyflow/classifiers/init.py:20: UserWarning: init: Could not import tiktorch classifier warnings.warn("init: Could not import tiktorch classifier") INFO ilastik_main: Starting ilastik from "/opt/ilastik". Starting ilastik from "/opt/ilastik". WARNING 2023-05-17 18:48:56,882 opConservationTracking 84 140041734346560 Could not find any ILP solver WARNING 2023-05-17 18:48:56,965 opStructuredTracking 84 140041734346560 Could not find any ILP solver WARNING 2023-05-17 18:48:56,969 structuredTrackingWorkflow 84 140041734346560 Could not find any learning solver. Tracking will use flow-based solver (DPCT). Learning for tracking will be disabled! INFO ilastik.shell.projectManager: Opening Project: pixelclassifier.ilp WARNING dataset.py(313): H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead. WARNING opSimpleBlockedArrayCache.py(43): FutureWarning: Conversion of the second argument of issubdtype from dtype to np.generic is deprecated. In future, it will be treated as `np.object == np.dtype(dtype).type. WARNING opSlicedBlockedArrayCache.py(141): FutureWarning: Conversion of the second argument of issubdtype fromdtypetonp.genericis deprecated. In future, it will be treated asnp.object_ == np.dtype(dtype).type`. ERROR 2023-05-17 18:49:01,873 log_exception 84 140041734346560 Traceback (most recent call last): File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 460, in _loadProject serializer.deserializeFromHdf5(self.currentProjectFile, projectFilePath, self._headless) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1151, in deserializeFromHdf5 ss.deserialize(topGroup) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationSerializer.py", line 100, in deserialize super().deserialize(group) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 264, in deserialize self._deserialize(group[self.name], self.inslot) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper return func(*args, **kwargs) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 622, in _deserialize self.inslot[index][slicing] = blockArray File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in setitem p[key] = value File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in setitem p[key] = value File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in setitem p[key] = value File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 988, in setitem self.operator.setInSlot(self.top_level_slot, self.subindex, roi, value) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 394, in setInSlot self._setInSlotInput(slot, subindex, roi, new_pixels) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 416, in _setInSlotInput block_rois = getIntersectingRois(self.Output.meta.shape, self._blockshape, (roi.start, roi.stop)) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 639, in getIntersectingRois block_rois = list(map(partial(getBlockBounds, dataset_shape, blockshape), block_starts)) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 667, in getBlockBounds block_bounds = getIntersection(block_bounds, entire_dataset_roi) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 402, in getIntersection assert ((stop - start) > 0).all(), "Rois do not intersect: {} and {}".format(roiA, roiB) AssertionError: Rois do not intersect: (array([0, 0, 1]), array([253, 252, 2])) and ([0, 0, 0], [500, 500, 1])

ERROR 2023-05-17 18:49:01,873 log_exception 84 140041734346560 Project could not be loaded due to the exception shown above. Aborting Project Open Action ERROR 2023-05-17 18:49:01,881 excepthooks 84 140041734346560 Unhandled exception in thread: 'MainThread' ERROR 2023-05-17 18:49:01,886 excepthooks 84 140041734346560 Traceback (most recent call last): File "/opt/ilastik/ilastik-meta/ilastik/ilastik.py", line 134, in main() File "/opt/ilastik/ilastik-meta/ilastik/ilastik.py", line 128, in main hShell = ilastik_main.main(parsed_args, workflow_cmdline_args) File "/opt/ilastik/ilastik-meta/ilastik/ilastik_main.py", line 170, in main f(shell) File "/opt/ilastik/ilastik-meta/ilastik/ilastik_main.py", line 347, in loadProject shell.openProjectFile(path, parsed_args.readonly) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/headless/headlessShell.py", line 125, in openProjectFile self.projectManager._loadProject(hdf5File, projectFilePath, readOnly) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper return func(*args, *kwargs) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 460, in _loadProject serializer.deserializeFromHdf5(self.currentProjectFile, projectFilePath, self._headless) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1151, in deserializeFromHdf5 ss.deserialize(topGroup) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationSerializer.py", line 100, in deserialize super().deserialize(group) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 264, in deserialize self._deserialize(group[self.name], self.inslot) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper return func(args, **kwargs) File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 622, in _deserialize self.inslot[index][slicing] = blockArray File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in setitem p[key] = value File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in setitem p[key] = value File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in setitem p[key] = value File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 988, in setitem self.operator.setInSlot(self.top_level_slot, self.subindex, roi, value) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 394, in setInSlot self._setInSlotInput(slot, subindex, roi, new_pixels) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 416, in _setInSlotInput block_rois = getIntersectingRois(self.Output.meta.shape, self._blockshape, (roi.start, roi.stop)) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 639, in getIntersectingRois block_rois = list(map(partial(getBlockBounds, dataset_shape, blockshape), block_starts)) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 667, in getBlockBounds block_bounds = getIntersection(block_bounds, entire_dataset_roi) File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 402, in getIntersection assert ((stop - start) > 0).all(), "Rois do not intersect: {} and {}".format(roiA, roiB) AssertionError: Rois do not intersect: (array([0, 0, 1]), array([253, 252, 2])) and ([0, 0, 0], [500, 500, 1])

[Milad4849]

Hi @eleonore-schneeg,

We typically see this type of error as a result of a mismatch between input data and probability maps/classification model of Ilastik. Could you provide some detail on what data you use, what you are trying to and what are the steps in your pipeline preceding the error?

[eleonore-schneeg]

I'm trying to do exactly the same thing as described in the manual. Everything runs from start to finish except that the ilastik step which has to be done from within the ilastik app and press the button 'batch processing'. The thing is that I would like to train a classifier on a few images and then be able to run it on a server on many images, so I would need the command line command to work.

These are the steps before the error:

steinbock preprocess imc images --hpf 50 steinbock classify ilastik prepare --cropsize 500 --seed 123 (keeping full size images instead of 50 in the example)

Opening ilastik manually and training the classifier It works if I press batch processing from within the app. steinbock classify ilastik run —> doesn’t work

[Milad4849]

Hi @eleonore-schneeg,

i have not been able to reproduce the error using my data so far, could you please confirm that you can run this script?

[eleonore-schneeg]

I ended up using the deep learning option instead, which works really well via the command line so it solved my issue, thanks!

I have a final question which is related to the command 'utils match' to intersect different masks. is it possible to intersect at the different 'thresholds' around the second object, to identify cells on but also + 10, +20, +... pixels around the second mask? If not is is possible to do so directly from the spatial object experiment on R? many thanks!

[Milad4849]

This is not an option with the current implementation. you can use steinbock utils match to find the matching objects in mask1 and mask2. then you can expand around objects in mask2 to find the neighbors within a given distance. This will give you correspondence between a given object in mask1 and objects within a given distance from its counterpart in mask2.

[eleonore-schneeg]

Right, I see how I could proceed that way but it would require to regenerate the masks multiple times. Are you aware of any way I could expand the objects on the spatial experiment object directly on R?

[nilseling]

Hi @eleonore-schneeg, could you please explain what exactly you want to do?

[eleonore-schneeg]

Basically I’m using the pipeline for IMC images to segment cells and quantify intensities in each cells. I’m investigating cells in the context of Alzheimer’s so I’m using additional masks of amyloid plaques and would like to analyse the cells at different distances around these plaques (ie starting on the plaques and then gradually increasing the distance). This way I can determine if distance to plaque has an effect on the behaviour of these cells.

[nilseling]

Ok, you can match the masks as mentioned before. Then you know which cells are part of the plaques. You can then use the minDistToCells function of imcRtools to calculate the distance of each cell to the plaque cells.

[eleonore-schneeg]

Oh that sounds perfect, thank you!!

[nilseling]

I'm closing this now. Please open a new issue if you run into problems.