search

DUNE / dist-comp

Action items for DUNE distributed computing, and common scripts that are used.
2 stars 0 forks source link

Add ability to force remove jobs from queue/schedule after a workflow has been finished by-hand #198

Open calcuttj opened 3 days ago

calcuttj commented 3 days ago

I'd like to remove submitted/running after their containing workflow has been finished (by hand). Not sure if the intended behavior is to have them operate 'normally' i.e. move through the statuses, but this is delaying further job submissions with fixes to the workflow requests.

For reference, workflow 4003 was finished at 2024-11-11 19:50:01 UTC but the jobs kept being started past this point, see the following images

image image

Andrew-McNab-UK commented 2 days ago

Thanks, I'll have to a look. I think the current behaviour is that no new jobs are submitted and no files are allocated to jobs after the workflow is marked finished. So a further improvement would be to tell the job to stop immediately as soon as it starts and asks for the jobscript. Beyond that we could kill the idle HTCondor jobs.