Determine which Cells exactly we need to compute the fused block, and prefetch every cell in a separate thread.
Add "--blocksPerJob" argument.
This specifies the "super-block multiplier", for example,
with "2,2,2" each spark job will output 2x2x2 neighboring blocks.
Prefetching and processing is done sequentially for each sub-block to avoid excessive memory usage for big super-blocks.
Refactoring:
Improve variable names. Add comments. Pull the Spark function lambda to separate class.
Fix bounding box math: We want to use the containing Interval (bigger than RealInterval) instead of contained Interval (smaller) for the transformed bounding box.
Determine which Cells exactly we need to compute the fused block, and prefetch every cell in a separate thread.
Add "--blocksPerJob" argument. This specifies the "super-block multiplier", for example, with "2,2,2" each spark job will output 2x2x2 neighboring blocks. Prefetching and processing is done sequentially for each sub-block to avoid excessive memory usage for big super-blocks.
Refactoring: Improve variable names. Add comments. Pull the Spark function lambda to separate class.
Fix bounding box math: We want to use the containing Interval (bigger than RealInterval) instead of contained Interval (smaller) for the transformed bounding box.