MoritzNegwer
commented
6 months ago Hi @Tas-V , thanks for reporting back! From your description, it seems you did set the folder structure correctly. It looks as if the downsample code either can't read or can't process your tiff files.
Here are a couple things you could try: 1) Naming conventions, the script expects files to end with .tif (not .tiff or so). If your files end in .tiff or so, you could try to bulk-rename them (or process via a python or Fiji load -> save as macro). 2) Metadata issues: We have seen similar behaviour in the past with some .ome.tif files when the header contained information that threw off the library we used to load. (Miltenyi/LaVision's Ultramicroscope control software Imspector used to produce those for a while, I think they fixed it now). If you have .ome.tif files, you could try to use Fiji to save them as normal tifs. (Depending on the settings the metadata might only be saved to the first image of the stack, so as a test you could try to remove the first image and replace it with a copy of the second image, then see whether this fixes the downsampling). 3) If you are okay with that, would you mind sharing the name of your ___slices folder and the name of one of the tiffs? That way we can try to reproduce the issue on our end, assuming it's a naming issue. 4) If you want a sanity-check, you could try to run the pipeline with our test dataset from https://discotechnologies.org/DELiVR/ , that is known to work on our setups at least.
I hope this helps. Best of luck!
Tas-V
I face this issue on both Windows and Ubuntu; here is my output from the Log of Fiji ImageJ:
`Loaded following JSON: { "raw_location" : "/data/raw/", "output_location" : "/data/output/", "mask_detection" : { "ilastik_location" : "/delivr/ilastik/", "ilastik_model" : "./models/random_forest_weights.ilp", "teraconverter_location" : "/delivr/teraconverter/", "output_location" : "/data/output/01_mask_detection/output/", "downsample_steps" : { "original_um_x" : 1.62, "original_um_y" : 1.62, "original_um_z" : 6.0, "downsample_um_x" : 25.0, "downsample_um_y" : 25.0, "downsample_um_z" : 25.0 }, "mask_with_Ilastik" : true, "simple_threshold_value" : 250 }, "blob_detection" : { "input_location" : "/data/output/01_mask_detection/output/", "model_location" : "./models/inference_weights.tar", "output_location" : "/data/output/02_blob_detection/output/", "window_dimensions" : { "window_dim_0" : 64, "window_dim_1" : 64, "window_dim_2" : 32 } }, "postprocessing" : { "input_location" : "/data/output/02_blob_detection/output/", "output_location" : "/data/output/03_postprocessing/output/", "min_size" : -1, "max_size" : -1 }, "atlas_alignment" : { "input_location" : "/data/output/03_postprocessing/output/", "output_location" : "/data/output/04_atlas_alignment/output/", "mBrainAligner_location" : "/delivr/mbrainaligner/", "collection_folder" : "/data/output/04_atlas_alignment/collection/", "parallel_processing" : true }, "region_assignment" : { "input_location" : "/data/output/04_atlas_alignment/collection/", "CCF3_atlasfile" : "./models/CCF3_P56_annotation.tif", "CCF3_ontology" : "./models/AllenMouseCCFv3_ontology_22Feb2021.xml", "output_location" : "/data/output/05_region_assignment/" }, "visualization" : { "input_csv_location" : "/data/output/05_region_assignment/", "input_size_location" : "/data/output/03_postprocessing/output/", "input_prediction_location" : "/data/output/02_blob_detection/output/", "cache_location" : "/data/output/06_visualization/cache/", "output_location" : "/data/output/06_visualization/output/", "region_id_rgb" : true, "region_id_grayvalues" : false, "no_atlas_depthmap" : false }, "FLAGS" : { "ABSPATHS" : false, "LOAD_ALL_RAM" : true, "TEST_TIME_AUGMENTATION" : true, "MASK_DOWNSAMPLE" : true, "BLOB_DETECTION" : true, "POSTPROCESSING" : true, "ATLAS_ALIGNMENT" : true, "REGION_ASSIGNMENT" : true, "VISUALIZATION" : true, "SAVE_MASK_OUTPUT" : true, "SAVE_NETWORK_OUTPUT" : true, "SAVE_ACTIVATED_OUTPUT" : false, "SAVE_POSTPROCESSING_OUTPUT" : true, "SAVE_ATLAS_OUTPUT" : true } } /home/taz/Documents/delivr_test/output/config.json Running docker run --rm -i --runtime=nvidia -v /home/taz/Documents/delivr_test/raw/:/data/raw/ -v /home/taz/Documents/delivr_test/output/:/data/output delivr:12 python3 main.py /data/output/config.json /home/taz/Documents/delivr_test/output /home/taz/Documents/delivr_test/output true
========== == CUDA ==
CUDA Version 11.7.1
Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License. By pulling and using the container, you accept the terms and conditions of this license: https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
Loading /data/output/config.json Transformed paths in config file. Making /data/output/04_atlas_alignment/collection/ For input string: "OVERALL" Masking Progress Overall: 17 Progress Task: 50 2024-04-23 15:23:18.677097 : Defining downsampling folders 2024-04-23 15:23:18.677756 : Downsampling 2024-04-23 15:23:18.845325 : Saving downsampled stacks Downsampled list empty! ERROR ::::::/usr/local/lib/python3.10/dist-packages/tifffile/tifffile.py:1606: UserWarning: <tifffile.TiffWriter 'stack_resampled.tif'> writing zero size array to nonconformant TIFF ERROR :::::: warnings.warn( ERROR ::::::Traceback (most recent call last): ERROR :::::: File "/delivr/main.py", line 94, in
ERROR :::::: downsample_mask(settings, brain)
ERROR :::::: File "/delivr/downsample/downsample_and_mask.py", line 223, in downsample_mask
ERROR :::::: downsampled_stack_8bit = histogram_equalization_8b(downsampled_stack)
ERROR :::::: File "/delivr/downsample/downsample_and_mask.py", line 123, in histogram_equalization_8b
ERROR :::::: minval = round(np.percentile(stack.ravel(),1))
ERROR :::::: File "<__array_function__ internals>", line 200, in percentile
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/lib/function_base.py", line 4205, in percentile
ERROR :::::: return _quantile_unchecked(
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/lib/function_base.py", line 4473, in _quantile_unchecked
ERROR :::::: return _ureduce(a,
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/lib/function_base.py", line 3752, in _ureduce
ERROR :::::: r = func(a, *kwargs)
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/lib/function_base.py", line 4639, in _quantile_ureduce_func
ERROR :::::: result = _quantile(arr,
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/lib/function_base.py", line 4745, in _quantile
ERROR :::::: take(arr, indices=-1, axis=DATA_AXIS)
ERROR :::::: File "<__array_function__ internals>", line 200, in take
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/core/fromnumeric.py", line 190, in take
ERROR :::::: return _wrapfunc(a, 'take', indices, axis=axis, out=out, mode=mode)
ERROR :::::: File "/usr/local/lib/python3.10/dist-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
ERROR :::::: return bound(args, **kwds)
ERROR ::::::IndexError: cannot do a non-empty take from an empty axes.
Done!
REGION PATH: /home/taz/Documents/delivr_test/output/05_region_assignment/
Opening Heatmaps...`
Here is my folder structure:
delivr_test ....................|raw ....................|..............|slices ....................|.................................|individual tiffs (z planes) ....................|output
I set the Raw Data Location to the raw folder and the output to output folder. I am unsure if I am missing anything else.