Describe the bug
When processing a large number of rasters (AOIs) >50 the processes (verification/tiling) becomes memory intensive. Hence, to process AOIs large amounts of memory are required which is not sustainable.
To Reproduce
Steps to reproduce the behavior:
Given large AOIs and constrained memory we run into OOM errors.
Expected behavior
We would like to successfully process large AOIs without OOM errors.
Screenshots
Additional context
The code logic in AOI.py needs to be rethought and refactored to allow for better memory management. e.g. refactor loops as generators.
Describe the bug When processing a large number of rasters (AOIs) >50 the processes (verification/tiling) becomes memory intensive. Hence, to process AOIs large amounts of memory are required which is not sustainable.
To Reproduce Steps to reproduce the behavior: Given large AOIs and constrained memory we run into OOM errors.
Expected behavior We would like to successfully process large AOIs without OOM errors.
Screenshots
Additional context The code logic in AOI.py needs to be rethought and refactored to allow for better memory management. e.g. refactor loops as generators.