search

flux-framework / flux-core

core services for the Flux resource management framework
GNU Lesser General Public License v3.0
167 stars 50 forks source link

python: need easy way to wait until a subinstance is ready to accept jobs #6178

Open grondo opened 3 months ago

grondo commented 3 months ago

When launching a subinstance via the Python api it is simple to wait for the memo event containing the URI and then open a child handle to the instance. If however, an attempt is immediately made to submit a job to the child instance, this will likely fail because the instance may not be ready. In order to ensure the instance is ready to accept jobs, a user needs to also wait for the state-machine.wait RPC to send a response.

We should make this simpler. Perhaps a way to post the URI memo only after state-machine.wait returns, or just a Python convenience API to do all this on behalf the user (e.g. return a child handle after the URI is available and state-machine.wait returns)

koparasy commented 3 months ago

@grondo I am just mentioning this here cause you refer to the memo event. It is indeed easy to get the uri through the memo event but there is no support to do that through the FluxExecutorFuture (at least I couldn't think of one) as memo is not part of MAIN_EVENTS (here) and the code raises an exception here . I have added some support for it here for my project not sure if any of it is useful for flux-core part. This though is kind of orthogonal to the state-machine.wait.