radical-cybertools / radical.pilot

RADICAL-Pilot
http://radical-cybertools.github.io/radical-pilot/index.html
Other
54 stars 23 forks source link

How would the Raptor instance cancel Tasks? #3002

Open eirrgang opened 1 year ago

eirrgang commented 1 year ago

There is not a complementary method to Raptor.submit_workers() . How would the Raptor instance cancel workers (or other Tasks)?

I would like to confirm the appropriate way for the Master to cancel a Task (worker or otherwise). If this is not supported (or not normative), I need to document that somewhere.

Note that my current expectation is that the ScalemsMaster needs to outlive the Worker(s) in order to clean up (handle final task disposition and workflow state). If this just seems unfeasible, then we need to rethink how we might clean up the filesystem artifacts and metadata (bookkeeping) from the client side after receiving confirmation that the Master and Worker(s) are shut down.

eirrgang commented 1 year ago

Update from the dev call: This is probably not a high priority issue. @andre-merzky will look into adding some API accessible to the Master. However, for the purposes of the scalems project, we can probably just shut down the Master (and, transitively, its workers) and do post hoc inspection of filesystem artifacts to determine any necessary cleanup or reconciliation of workflow state.