saalfeldlab / n5-ij

ImageJ convenience layer for N5
BSD 2-Clause "Simplified" License
14 stars 8 forks source link

Opening N5 is slow #59

Open StephanPreibisch opened 1 year ago

StephanPreibisch commented 1 year ago

Hi, for me opening only uses a single thread more or less, which is not intended. Thus I was wondering how the LoopBuilder distributes tasks: https://github.com/saalfeldlab/n5-ij/blob/ba707bae18152e4293e8faf749d703db61a8edd9/src/main/java/org/janelia/saalfeldlab/n5/ij/N5Importer.java#L487-L489

Are maybe all processed pixels close to each other, so it wouldn't really open different blocks in parallel since all threads wait for the same block to be loaded?

Also, for virtual loading one should use multi-threading as well. ImageJFunctions.show does take a ExecutorService optionally for that ... we wrote this for BigStitcher some time ago. Because even if you load one z-plane virtually, you still need to load the entire blocks that are usually big in z.

Happy to help, Stephan