Closed AnthonyChanMarvel closed 3 years ago
Version of deepflash2 is 0.1.3
Hi @AnthonyChanMarvel , if you're doing the Hubmap challenge you need to monkey patch the read_img
method - see https://www.kaggle.com/matjes/hubmap-efficient-sampling-deepflash2-train
e.g.
@patch
def read_img(self:BaseDataset, file, *args, **kwargs):
return zarr.open(str(file), mode='r')
Does this fix the bug?
@matjesg
I get an unexpecetd error from learn.predict tiles. I used different loss func than CE. please help
/opt/conda/lib/python3.7/site-packages/deepflash2/learner.py in predict_tiles(self, ds_idx, dl, path, mc_dropout, n_times, use_tta, tta_merge, tta_tfms, uncertainty_estimates, energy_T)
228 z_eng = g_eng.empty(f.name, shape=outShape, dtype='float32')
229 last_file = f
--> 230 z_smx[outSlice] = smx[inSlice]
231 z_seg[outSlice] = np.argmax(smx, axis=-1)[inSlice]
232 if uncertainty_estimates:
/kaggle/input/zarrkaggleinstall/zarr/core.py in __setitem__(self, selection, value)
1120
1121 fields, selection = pop_fields(selection)
-> 1122 self.set_basic_selection(selection, value, fields=fields)
1123
1124 def set_basic_selection(self, selection, value, fields=None):
/kaggle/input/zarrkaggleinstall/zarr/core.py in set_basic_selection(self, selection, value, fields)
1215 return self._set_basic_selection_zd(selection, value, fields=fields)
1216 else:
-> 1217 return self._set_basic_selection_nd(selection, value, fields=fields)
1218
1219 def set_orthogonal_selection(self, selection, value, fields=None):
/kaggle/input/zarrkaggleinstall/zarr/core.py in _set_basic_selection_nd(self, selection, value, fields)
1506 indexer = BasicIndexer(selection, self)
1507
-> 1508 self._set_selection(indexer, value, fields=fields)
1509
1510 def _set_selection(self, indexer, value, fields=None):
/kaggle/input/zarrkaggleinstall/zarr/core.py in _set_selection(self, indexer, value, fields)
1534 if not hasattr(value, 'shape'):
1535 value = np.asanyarray(value)
-> 1536 check_array_shape('value', value, sel_shape)
1537
1538 # iterate over chunks in range
/kaggle/input/zarrkaggleinstall/zarr/util.py in check_array_shape(param, array, shape)
514 if array.shape != shape:
515 raise ValueError('parameter {!r}: expected array with shape {!r}, got {!r}'
--> 516 .format(param, shape, array.shape))
517
518
ValueError: parameter 'value': expected array with shape (924, 924, 2), got (924, 924, 1)
@matjesg FYI: I faced the exact same error as Antony above, tried with @patch
and versions 0.1.2, 0.1.3
Also if I define the datasets as:
train_ds = RandomTileDataset(files, **ds_kwargs)
valid_ds = TileDataset(files, **ds_kwargs, is_zarr=True)
then works ok. Tracing the error has something to do with reading masks and possibly skimage
(?) Any help would be much appreciated
@jaideep11061982 Unfortunately, the current release of deepflash2 does not support single class predictions - it will be available in the next release. Until then, you patch the 'predict_tiles()' method (https://github.com/matjesg/deepflash2/blob/master/deepflash2/learner.py#L212) by adding the following code after line 212
if dl.c==1:
out_act = torch.sigmoid(out)
out_act = torch.cat([(1-out_act), out_act], dim=1)
(The code in the 'tile_shift' branch (https://github.com/matjesg/deepflash2/blob/tile_shift/deepflash2/learner.py#L244) already supports single class prediction, but also adds some more features that would probably break the current release.)
Hope this fixes your problem for now :)
@i-mein
Does your problem also relate to the Hubmap challenge? It is hard for me to track down the errors, but the data there needs custom preprocessing (as shown in my notebooks, via the tifffile
package) before it can be loaded into the deepflash2 datasets. I hope this will fix the problem!
Yes it is about Hubmap and your example notebook, I use images from hubmap-zarr
(v.11) and masks from hubmap-labels-pdf-0-5-0-25-0-01
(v.4) without any other change but the error remains. I suspect something with wrong path on reading masks since when I remove label_fn works OK (but loads only images of course) - I try to found a workaround and let you know
ps: I spend a lot of time reading the gist and the tutorials and It's pitty I can't run any experiments
Thanks for your help
EDIT: The exact error I got is the same ValueError
as shown in the image above.
Just to let you know that finally I make it work. Was either a path or a version messed up - when I started a new clean env and install latest version (0.1.3) problem fixed.
Thanks for the help
I tried to use RandomTileDataset and TileDataset in my pipeline, but TileDataset returned an error Is this a dependency problem?