Open clane9 opened 1 week ago
.get_fdata()
casts to a floating point dtype
, float64
by default. You may prefer just np.asanyarray(img.dataobj)
, which will ensure that scaling is applied but preserve the memmap if no scaling is necessary.
If you know the dtype
you want to operate in, np.int16img.dataobj)
(for example) would work. If you're using targeting float32
, img.get_fdata(dtype='float32')
will work and cache the result for the next call.
Thanks @effigies, I think these are good strategies for interacting with the memmap data. But still I'm confused why the simple get_fdata()
is so much slower when mmap=True
(the default)?
Regardless, I will just set mmap=False
for my case and avoid the issue, so it is not too critical to get to the bottom of.
I don't really know why the memmap is slower. I would probably profile it and see where it's spending the time.
Ya I did a quick profiling, with mmap=True
, it's spending almost all of the time here, doing casting it seems.
With mmap=False
, the casting takes less time and most of the time is spent doing the read
I guess when the array is a memmap, astype()
does an implicit read, which is slower for some reason than the explicit read when mmap=False
.
I'm not sure there's much to be done here. That said, if you're up for it, you could try looking into and benchmarking other approaches to scaling. e.g., pre-allocating an array with np.empty(dtype=target_dtype)
and assigning its values with np.copyto.
That will move more of the branching logic (e.g., are dtypes the same) into nibabel, and it has been nice to let numpy make those decisions and trust them to be efficient. It could be worth upstreaming the solution (if we find one) into numpy's memmap.astype()
implementation.
Ya I agree, I feel like there's not much worth doing at this point. Especially since in this case mmap=False
is fine. Just wanted to make a note since it did confuse us for a little while before we tracked it down. Will let you know if we end up feeling motivated and decide to look for some improvement.
It seems that reading uncompressed nifti volumes takes significantly longer when memory mapping is enabled. In this example, I load four large 4D nifti volumes from the ABCD dataset, with and without memory mapping.
This is what I get, using nibabel v5.2.1:
Any idea why this might happen?