flux-framework / flux-core

core services for the Flux resource management framework
GNU Lesser General Public License v3.0
167 stars 49 forks source link

job-info: single RPC that can handle updated lookups and normal lookups #5577

Open chu11 opened 10 months ago

chu11 commented 10 months ago

Problem: The current python API functions that call job-info.lookup can take multiple keys for lookup. Users may want to get the keys for an updated value (i.e. R w/ eventlog updates) and other non-updated key values in one RPC call.

Doing this would require doing a RPC to job-info.lookup and job-info.update-lookup. While is perfectly doable, it's certainly annoying to do.

So I began thinking, should there be a single RPC that can handle both?

easiest idea: update job-info.update-lookup to handle looking up any job key, mostly likely by forwarding all keys to job-info.lookup (and only handling "update" ones specially before returning things to the caller)

I thought this was a super obvious / good idea, until I realized ... most people might want to always call job-info.update-lookup instead of job-info.lookup ...

So ... maybe it'd be better if job-info.lookup had a flag to handle "updated" keys instead?

Hindsight being 20/20, job-info.update-lookup was sort of "added on" after job-info.update-watch was written. So admittedly some things were perhaps not thought out super well in the end.

chu11 commented 9 months ago

after working on #5593, I decided that making job-info.update-lookup take any key was not worth it, as there were many subtleties underneath the covers.

I could close this, as I think having a generic "wrapper" service could be useful. However, it's not super necessary given it can be hidden via code in the python bindings for the time being.

garlick commented 9 months ago

If it's helpful from a design standpoint, I could throw together an RFC for the "classic" job-info RPCs

Then maybe extensions for the equivalent to "update-lookup" and "update-watch" could be first proposed as updates to that RFC?

chu11 commented 9 months ago

Sounds like a good idea, minimally it's good to document the "classic" RPCs by themselves