After implementing ModelLibrary in calwebb_image3 (see [JP-3690[https://jira.stsci.edu/browse/JP-3690]), the biggest memory bottleneck in the image3 pipeline is resample_many_to_one. The reason for this is the requirement to allocate a data array, three separate variance arrays, a weighting array, and a context array, all of which are fairly large as they scale with the size of the output.
Two options could be added to resample to reduce memory usage:
Make computation of the context array optional. This PR in drizzle makes it possible to set enable_context to False, and this option could also be exposed in resample.
Make computation of the variance arrays optional with a parameter to resample.
The use-cases for these options, their defaults, and the benefits/drawbacks of exposing them need to be discussed with INS. Once a decision is made, this ticket will cover that work.
Issue JP-3707 was created on JIRA by Ned Molter:
After implementing ModelLibrary in calwebb_image3 (see [JP-3690[https://jira.stsci.edu/browse/JP-3690]), the biggest memory bottleneck in the image3 pipeline is resample_many_to_one. The reason for this is the requirement to allocate a data array, three separate variance arrays, a weighting array, and a context array, all of which are fairly large as they scale with the size of the output.
Two options could be added to resample to reduce memory usage:
The use-cases for these options, their defaults, and the benefits/drawbacks of exposing them need to be discussed with INS. Once a decision is made, this ticket will cover that work.