Open tiemvanderdeure opened 3 weeks ago
What is tas in your example?
Sorry, tas
should be ser
Ok looks like some problem like reading every single cell separately.... it should just be one chunk looking at you eachchunk
output. So the ConcatDiskArray
isn't doing its job properly.
It might be fixed on DiskArrays main but we aren't using it yet because of NCDatasets
This is on DiskArrays v0.4.4. I don't have NCDatasets in the environment
Ok... so the problem could be that we need to special-case ConCatDiskArray and just open the files that it needs to read from? They wont be found by Flatten.flatten
because theyre objects in an array.
I had assumed readblock
would do that fine because each block is a whole file and it would only open once... but something is not working and we are reading into a FileArray
many times instead of opening it once.
MWE:
This must be some issue with chunks and getindex.
All the time is being spent on![image](https://github.com/user-attachments/assets/23a53262-a4de-4dbe-98e5-119b97121f6e)
gdalopen
andrasterio!
andgdalclose