search

dirac-institute / kbmod

KBMOD (Kernel-Based Moving Object Detection)
BSD 2-Clause "Simplified" License
40 stars 14 forks source link

Reproject taking 300+ Gb of RAM for 200 (mosaics) #615

Open ColinOrionChandler opened 1 month ago

ColinOrionChandler commented 1 month ago

Too much RAM

wilsonbb commented 1 month ago

To add context, I was seeing failures here while using a WorkUnit with 876 images before mosaicing happen.

Each image probably had 3 layers that were around 55.5 mb each. So if all of the images were concurrently loaded into RAM for the WorkUnit, that's ~145 GB

At the pre-mosaic stage however, we probably do not need all images in memory at the same time