Open ajstewart opened 1 year ago
Turns out that in order to delete the run, django is likely loading all objects into memory.
Currently the deletion logic is a simple:
https://github.com/askap-vast/vast-pipeline/blob/f2c207cd1b40e9a5e50ecba6e9c6468fa3da393d/vast_pipeline/management/commands/clearpiperun.py#L115
and with the cascading effect this is likely causing all sources and image objects to be loaded into memory.
Fix will be to break down the logic to delete components over a generated list.
This is likely the cause of the woes of Nimbus over the past few months because of the stuck command in the qcluster queue.
Turns out that in order to delete the run, django is likely loading all objects into memory.
Currently the deletion logic is a simple:
https://github.com/askap-vast/vast-pipeline/blob/f2c207cd1b40e9a5e50ecba6e9c6468fa3da393d/vast_pipeline/management/commands/clearpiperun.py#L115
and with the cascading effect this is likely causing all sources and image objects to be loaded into memory.
Fix will be to break down the logic to delete components over a generated list.
This is likely the cause of the woes of Nimbus over the past few months because of the stuck command in the qcluster queue.