search

TomographicImaging / CIL

A versatile python framework for tomographic imaging
https://tomographicimaging.github.io/CIL/
Apache License 2.0
98 stars 45 forks source link

TIFFStackReader speed #2000

Open adamdoc opened 3 days ago

adamdoc commented 3 days ago

Description

TIFFStackReader is a lot slower than the max read speed of my given hardware

For a 17.6GB dataset: TIFFStackReader takes roughly 100sec (180MB/s) ImageJ takes 30sec (600MB/s) PIL in python takes 61sec (300MB/s)

The data is on a SSD on a PCIe slot on the motherboard, so the 180MB/s speed is very slow. I've recently started using 100GB datasets, so this equates to near 10mins of data loading time which can be quite tedious for testing different configurations

Environment

cil 24.2.0 np126py312_0 ccpi

gfardell commented 3 days ago

Thanks @adamdoc is this on windows or Linux?

It's certainly something we can look in to!

adamdoc commented 11 hours ago

Thanks, I'm on Windows 11