Closed bwtan closed 4 years ago
The script is called Tubemap.py in the Scripts fodder.
The script is called Tubemap.py in the Scripts fodder.
Thanks! Can I assume this Clearmap2 still runs in linux system (ubuntu)? What parts that may need to be tweaked if using windows?
I get this error message when I try to run the script:
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/TIF.py", line 72, in array
array = self._tif.asarray(validate = False, maxworkers=processes);
TypeError: asarray() got an unexpected keyword argument 'validate'
I get this error message when I try to run the script:
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/TIF.py", line 72, in array array = self._tif.asarray(validate = False, maxworkers=processes); TypeError: asarray() got an unexpected keyword argument 'validate'
I also got this error message. Did you manage to fix it?
Yes we fixed it, please wait for the new version and Documentation to get online very soon.
Am Mi., 10. Juni 2020 um 14:03 Uhr schrieb bwtan95 <notifications@github.com
:
Reopened #2 https://github.com/ChristophKirst/ClearMap2/issues/2.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChristophKirst/ClearMap2/issues/2#event-3431457844, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMCTWVWXOV2J7BHT3RWENDRV7YJZANCNFSM4NESWNWA .
The new version should fix this issue. If not please reopen this issue.
Now I get this error:
Traceback (most recent call last):
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-788a7e8ab86d>", line 1, in <module>
runfile('/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py', wdir='/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts')
File "/snap/pycharm-professional/201/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/snap/pycharm-professional/201/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py", line 84, in <module>
layout = stw.WobblyLayout(expression=ws.filename('raw'), tile_axes=['X','Y'], overlaps=(45, 155));
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Alignment/Stitching/StitchingWobbly.py", line 725, in __init__
strg.TiledLayout.__init__(self, sources = sources, expression = expression, tile_axes = tile_axes, tile_shape = tile_shape, tile_positions = tile_positions, positions = positions, overlaps = overlaps, alignments = alignments, position = position, shape = shape, dtype = dtype, order = order);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Alignment/Stitching/StitchingRigid.py", line 1909, in __init__
sources, alignments, tile_positions = _initialize_tiles_from_expression(expression, tile_axes=tile_axes, tile_shape=tile_shape, tile_positions=tile_positions, overlaps=overlaps, positions=positions, alignments=alignments);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Alignment/Stitching/StitchingRigid.py", line 2270, in _initialize_tiles_from_expression
tile_shape = np.max(tile_positions, axis = 0) + 1; # assume that numbering starts with 0!
File "<__array_function__ internals>", line 6, in amax
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2668, in amax
keepdims=keepdims, initial=initial, where=where)
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 90, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation maximum which has no identity
Does it rely on files which were not published together with the code?
TubeMap is a data analysis script for TB data sets. We do not include those in our software. We have sample data here: https://osf.io/sa3x8/
Many thanks for the link! Are there perhaps besides the complete brains also small toy examples for quick testing of the pipeline?
I agree with @saskra, can we get a small toy example with maybe 4-stacks?
Each .tif on osf.io is 3.69 GB with (3501, 404, 1302) pixels. I can't run the entire pipeline on my non super-computer machine. If I could it would take way too long to play with different parameters in the code.
In particular, I am trying to play with Scripts/TubeMap.py
. I am most interested in per-stack algorithms to generate a mask of vasculature. Not as interested in the stitching.
Can we get a toy example that can run on a consumer machine? If I understand the osf.io website, the dataset has been downloaded just 2 times. By providing a smaller dataset, maybe 10+ or 20+ people/labs would give your code a try?
Hi,
thanks for the feed back.
We can provide a smaller data set at some point together with some tests etc.
In the mean time, you can always just download a single tile (from the center of the brain ideally) from our data set and work with that. If that is still too big, you can also easily cut out a smaller subset.
There is a debug/test mode for ClearMap does that for you (see the tutorial)
If you prefer to work with a small data set directly, you can simply use this code
import ClearMap.IO.IO as io
source = io.as_source('filename_of_tile.tif');
subset = source[500:1000, 500:1000, 500:1000];
io.write('filename_of_subset.npy', subset);
to generate a small block (here 500**3) of data and work with that. It should be small enough for a standard computer to work with.
To run the tutorial then simply modify the file names accordingly and skip the stitching part in the tutorial.
Hope this helps.
Out of curiosity: This code is designed to work for large 3d images but I would be curious what smaller data sets you are planning to use ClearMap with ? It is possible to add another expert pipeline for other data and we are currently working on analyzing super resolution / expansion microscopy data sets.
Christoph
Out of curiosity: This code is designed to work for large 3d images but I would be curious what smaller data sets you are planning to use ClearMap with ?
We have many "small" 3D ROIs and are therefore interested in automatizable and fast algorithms for segmentation and skeletonization.
In the mean time, you can always just download a single tile (from the center of the brain ideally) from our data set and work with that.
Thanks for the tip! So I took the autofluorescence from the first brain 39L completely and from the raw data only four consecutive files from the middle (Y02, X05-06, Filter0000-0001).
#directories and files
directory = '/home/saskra/PycharmProjects/ClearMap2/ClearMap/Tests/Data/TubeMap_Example'
expression_raw = 'Raw/14-16-41_tricocktail_UltraII[<Y,2> x <X,2>]_C00_UltraII Filter0001.ome.tif'
expression_arteries = 'Raw/14-16-41_tricocktail_UltraII[<Y,2> x <X,2>]_C00_UltraII Filter0000.ome.tif'
expression_auto = 'Autofluorescence/14-02-13_auto_UltraII_C00_xyz-Table Z<Z,4>.ome.tif'
Unfortunately the script only runs until this error message:
Traceback (most recent call last):
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-788a7e8ab86d>", line 1, in <module>
runfile('/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py', wdir='/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts')
File "/snap/pycharm-professional/201/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/snap/pycharm-professional/201/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py", line 184, in <module>
"result_directory" : ws.filename('elastix_resampled_to_auto')
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/Workspace.py", line 286, in filename
debug=debug, **kwargs);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/Workspace.py", line 152, in filename
raise ValueError('Cannot find name for type %r!' % ftype);
ValueError: Cannot find name for type 'elastix_resampled_to_auto'!
This was solved with one of the last commits.
Next error, but I might open a new issue for that one:
Graph reduction: initialized.
Graph reduction: Found 20393 branching and 592581 non-branching nodes: elapsed time: 0:00:00.591
Graph reduction: Scanned 20393/20393 branching nodes, found 1018 branches: elapsed time: 0:00:01.420
Graph reduction: Scanned 250000/592581 non-branching nodes found 11667 branches: elapsed time: 0:00:09.205
Graph reduction: Scanned 500000/592581 non-branching nodes found 22705 branches: elapsed time: 0:00:16.715
Graph reduction: Scanned 592581/592581 non-branching nodes found 26871 branches: elapsed time: 0:00:19.219
Graph reduction: Graph reduced from 612974 to 20393 nodes and 619452 to 26871 edges: elapsed time: 0:00:20.528
Transforming vertex property: coordinates -> coordinates_atlas
Traceback (most recent call last):
File "/snap/pycharm-professional/209/plugins/python/helpers/pydev/pydevd.py", line 1438, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/snap/pycharm-professional/209/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py", line 482, in <module>
verbose=True);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Analysis/Graphs/GraphGt.py", line 1366, in transform_properties
values = transformation(values);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/Scripts/TubeMap.py", line 465, in transformation
sink_shape=io.shape(ws.filename('resampled')));
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/IO.py", line 258, in shape
source = as_source(source);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/IO.py", line 206, in as_source
source = mod.Source(source, *args, **kwargs);
File "/home/saskra/PycharmProjects/ClearMap2/ClearMap/IO/TIF.py", line 36, in __init__
self._tif = tif.TiffFile(location, multifile = multi_file);
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/tifffile/tifffile.py", line 2183, in __init__
fh = FileHandle(arg, mode='rb', name=name, offset=offset, size=size)
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/tifffile/tifffile.py", line 6408, in __init__
self.open()
File "/home/saskra/anaconda3/envs/ClearMap/lib/python3.7/site-packages/tifffile/tifffile.py", line 6421, in open
self._fh = open(self._file, self._mode)
FileNotFoundError: [Errno 2] Datei oder Verzeichnis nicht gefunden: '/home/saskra/PycharmProjects/ClearMap2/ClearMap/Tests/Data/TubeMap_Example/debug_resampled.tif'
Hi Christoph,
Could you share a copy of the main script running through the whole pipeline?