Open braingram opened 8 months ago
stdatamodels currently provides check_memory_allocation: https://github.com/spacetelescope/stdatamodels/blob/e1ebc483f574f03057e2474bb5cbef38f101735c/src/stdatamodels/jwst/datamodels/util.py#L392 which is designed to check the (ideally estimated) memory consumption of a model against the available memory. This is useful for steps like resample and outlier_detection in both jwst and romancal to support the allowed_memory option: https://jwst-pipeline.readthedocs.io/en/latest/jwst/outlier_detection/arguments.html#step-arguments-for-non-ifu-data The romancal code is not using stdatamodels (as it does not depend on that package). As mentioned https://github.com/spacetelescope/jwst/pull/8324#issuecomment-1975240069 stcal might be a suitable location for some shared code to check estimated memory against available memory during processing.
check_memory_allocation
resample
outlier_detection
jwst
romancal
allowed_memory
stdatamodels
stcal
stdatamodels currently provides
check_memory_allocation: https://github.com/spacetelescope/stdatamodels/blob/e1ebc483f574f03057e2474bb5cbef38f101735c/src/stdatamodels/jwst/datamodels/util.py#L392 which is designed to check the (ideally estimated) memory consumption of a model against the available memory. This is useful for steps likeresampleandoutlier_detectionin bothjwstandromancalto support theallowed_memoryoption: https://jwst-pipeline.readthedocs.io/en/latest/jwst/outlier_detection/arguments.html#step-arguments-for-non-ifu-data Theromancalcode is not usingstdatamodels(as it does not depend on that package). As mentioned https://github.com/spacetelescope/jwst/pull/8324#issuecomment-1975240069stcalmight be a suitable location for some shared code to check estimated memory against available memory during processing.