After some initial performance tests with XRay, I can see unsurprisingly IO consumes the most time. Currently all the rasters are being read one at time. One easy optimization we could do is just to read all the input rasters in parallel upfront.
A simple experiment on a 1 MHa geometry using the threading module showed it works pretty well. Before, each read took ~2 seconds, one after another. Multithreaded, as expected, it takes ~2 seconds total to read all at the same time.
After some initial performance tests with XRay, I can see unsurprisingly IO consumes the most time. Currently all the rasters are being read one at time. One easy optimization we could do is just to read all the input rasters in parallel upfront.
A simple experiment on a 1 MHa geometry using the threading module showed it works pretty well. Before, each read took ~2 seconds, one after another. Multithreaded, as expected, it takes ~2 seconds total to read all at the same time.