GeoscienceAustralia / PyRate

A Python tool for estimating velocity and time-series from Interferometric Synthetic Aperture Radar (InSAR) data.
https://geoscienceaustralia.github.io/PyRate/
Apache License 2.0
200 stars 70 forks source link

Replace dictionaries with buffers when merging loop closure calculations between processes [ci skip] #339

Closed richardt94 closed 3 years ago

richardt94 commented 3 years ago

Was having another issue with sum_phase_closures which was related to the numpy-array-in-a-dictionary design pattern we're using for inter-process communication - this simply doesn't work with big stacks of interferograms because if the limit on maximum size of a pickle, so I replaced it with a preallocated buffer and a call to Gatherv, along with the associated calculations of buffer sizes and stuff for each process.