Extraction from running extract on Dante two tiles:
It was 132 images of shape 2304x2304x50 at uint16. This is 70GB in total of disk reading.
The extract time was 170 minutes. This corresponds to a maximum of 2,000GB of reading (server reads are at 200MB/s on used device).
2,000GB / 70GB is 29. This is close to 28 channels. There are 28 channels in each ND2 file.
So, I think extraction is slow because everytime you gather just one channel image, it gathers all 28 channels.
To fix this, I must look into the ND2 library soon and try gather all channels for a round in one batch.
This way we do not have to keep loading in every channel just to pick out single channels.
Extraction from running extract on Dante two tiles:
It was 132 images of shape 2304x2304x50 at uint16. This is 70GB in total of disk reading.
The extract time was 170 minutes. This corresponds to a maximum of 2,000GB of reading (server reads are at 200MB/s on used device).
2,000GB / 70GB is 29. This is close to 28 channels. There are 28 channels in each ND2 file.
So, I think extraction is slow because everytime you gather just one channel image, it gathers all 28 channels.
To fix this, I must look into the ND2 library soon and try gather all channels for a round in one batch. This way we do not have to keep loading in every channel just to pick out single channels.