Open ColinDTaylor opened 4 years ago
Same problem for me. Any updates?
As an alternative, pims_nd2
handles the same dataset without problems.
Hi Ruben (@rbnvrw ), I'm having a similar issue with a stitch image:
ValueError: cannot reshape array of size 1038565 into shape (1019,1019)
metadata:
{'height': 1019, 'width': 1019, 'date': datetime.datetime(2017, 6, 6, 11, 15, 6), 'fields_of_view': range(0, 1), 'frames': [0], 'z_levels': range(0, 21), 'z_coordinates': None, 'total_images_per_channel': 21, 'channels': ['CSU far red RNA', 'CSU red RNA', 'CSU green RNA', 'CSU blue RNA', 'CSU BF'], 'pixel_microns': 1.0153102411107793, 'num_frames': 1, 'experiment': {'description': 'ND Acquisition', 'loops': []}, 'events': []}
Dimension:
Dimensions: XY(1) x λ(5) x Z(21)
What I found in
def _get_raw_image_data(self, image_group_number, channel_offset, height, width):
[...]
image_group_data[image_data_start::number_of_true_channels]
the slicing is good for the first 5(channels) x 1019 pixels then there is an extra 0 that is added to the data that need to be removed, I added a very crude:
unwanted = []
for i in range(5099, 5192828, 5095):
unwanted.append(i)
for ele in unwanted:
del image_group_data[ele]
5099 is 1019x5+4(image_data_start), 5192828 is the length of image_group_data and 5095 is 1019x5. Obviously it was just me trying to figure out what was wrong. I wonder if the XY(1) is Dimensions is an indication that something is wrong, but, unfortunately, I don't have more stitched images to try.
Thank you for the addition Cedric. @bioimage-analysis @ColinDTaylor @aaristov could you maybe all add a link to a sample ND2 file that I can download? I'd really like to solve this, but I don't have any files I can try to analyze. Thanks! :+1:
Hey @rbnvrw, mine is the sample_data from: https://downloads.openmicroscopy.org/images/ND2/karl/
This should now be fixed in the latest master version with PR #40 (thanks again @ggirelli !) If anyone could test this with their images, that would be appreciated.
Amazing! Now I can read my stitched nd2 files! Thanks a lot guys @rbnvrw @ggirelli ! Please bump the version number to signify this fix.
The fix seems to break things for 3D multi-channel images... I am looking into it.
@ggirelli OK thank you. please keep us posted. Does it happen for images with (x,y,z,c) channels only or also (x,y,z,t) or (x,y,c,t) images for example?
@ggirelli could you test this again with the latest fix from #50?
@rbnvrw sorry, I am back after a short break after submitting my thesis :) I'll test this later this week and post here about it!
@ggirelli congratulations on submitting your thesis! do you have time to test this now? :-)
Hi @rbnvrw , thanks and sorry, it's been crazy a crazy period before/after the defense :) I will take care of these in the next couple of weeks!
@rbnvrw better late than ever, I just tested it on my files. All good on my end 😄 🎉 🎆
Hi guys!
So I hate to ruin the good mood but I am running into a similar issue.
I'll copy-paste it here it for convenience and it's completely fine for me to share the file (36.6 MB) if you can help me. I saw you might want the metadata from NIS Viewer, I'll see if I can locate it.
Interestingly it is only one of the two channels that has the issue (so I can use images[0]
as normal but iterating or images[1]
will trigger the ValueError
).
Some nd2 images has images seem to miss one data point, making it impossible to reshape their array which also makes the file non-iterable.
Notably
18105583 == 18105584 - 1
, where18105584 == 3892 * 4651
so it would seem one data point is missing.NIS-viewer can open the file without issues.
Error
ValueError: cannot reshape array of size 18105583 into shape (3892,4651)
Code to reproduce
#! /bin/python3 from nd2reader import ND2Reader INPUT = "/Users/tobiasfrick/PycharmProjects/demos/test-data/multiple-channels.nd2" def iterate(): with ND2Reader(INPUT) as images: for _ in images: pass if __name__ == '__main__': iterate()
Versions
% python --version Python 3.8.10 % conda list | grep nd2reader nd2reader 3.3.0 pyhd8ed1ab_0 conda-forge
Traceback
python main.py Traceback (most recent call last): File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/parser.py", line 277, in _get_raw_image_data image_data = np.reshape(image_group_data[image_data_start::number_of_true_channels], (height, width)) File "<__array_function__ internals>", line 5, in reshape File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 298, in reshape return _wrapfunc(a, 'reshape', newshape, order=order) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 54, in _wrapfunc return _wrapit(obj, method, *args, **kwds) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 43, in _wrapit result = getattr(asarray(obj), method)(*args, **kwds) ValueError: cannot reshape array of size 18105583 into shape (3892,4652) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "main.py", line 18, in <module> main() File "main.py", line 13, in main for i in images: File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/slicerator/__init__.py", line 227, in <genexpr> return (self._get(i) for i in self.indices) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/slicerator/__init__.py", line 207, in _get return self._ancestor[key] File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/slicerator/__init__.py", line 188, in __getitem__ return self._get(indices) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/pims/base_frames.py", line 98, in __getitem__ return self.get_frame(key) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/pims/base_frames.py", line 592, in get_frame result = self._get_frame_wrapped(**coords) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/reader.py", line 88, in get_frame_2D return self._parser.get_image_by_attributes(t, v, c, z, y, x) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/parser.py", line 103, in get_image_by_attributes timestamp, raw_image_data = self._get_raw_image_data(image_group_number, channel, File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/nd2reader/parser.py", line 280, in _get_raw_image_data image_data = np.reshape(image_group_data[image_data_start::number_of_true_channels], (height, new_width)) File "<__array_function__ internals>", line 5, in reshape File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 298, in reshape return _wrapfunc(a, 'reshape', newshape, order=order) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 54, in _wrapfunc return _wrapit(obj, method, *args, **kwds) File "/Users/tobiasfrick/miniconda3/envs/demos/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 43, in _wrapit result = getattr(asarray(obj), method)(*args, **kwds) ValueError: cannot reshape array of size 18105583 into shape (3892,4651)
I'm getting the same error for any image I have taken where I am using more than one channel. So possibly this merits a separate issue as I can reproduce it even when I have non-stitched images.
For every image I test on from my most recent dataset, channel=0
always works whereas channel=1
never works (for example in a syntax like this image = images.get_frame_2D(c=channel)
).
Hello Ruben,
I am having an issue that I noticed had come up some time in the past but was never fully resolved (in https://github.com/rbnvrw/nd2reader/issues/6). When trying to open certain .nd2 files using nd2Reader I get the error:
ValueError: cannot reshape array of size 45686334 into shape (6759,6759)
This happens only on certain images from this run, and I can't seem to find any way to predict which images will open alright and which will not. We recently changed the stitching in Elements to overlap slightly more than it used to, and maybe that has some connection to this problem cropping up again?
Here is the object info:
<FramesSequenceND> Axes: 4 Axis 'x' size: 6759 Axis 'y' size: 6759 Axis 'c' size: 3 Axis 't' size: 1 Pixel Datatype: <class 'numpy.float64'>
bundle_axes:
['y', 'x']
iter_axes:
['c']
metadata:
{'height': 6759, 'width': 6759, 'date': datetime.datetime(2019, 10, 29, 13, 0, 1), 'fields_of_view': [0], 'frames': [0], 'z_levels': [], 'z_coordinates': [5246.4125], 'total_images_per_channel': 1, 'channels': ['Hoechst', 'AF488', 'AF555'], 'pixel_microns': 0.648678997775168, 'num_frames': 1, 'experiment': {'description': 'unknown', 'loops': []}, 'events': []}
Here is a traceback: