Add rabbit entries to `flux job info` list of useful keys

[jameshcorbett]

flux-coral2 software adds a number of entries to jobs' KVS (see https://flux-framework.readthedocs.io/en/latest/tutorials/lab/rabbit.html#additional-attributes-of-rabbit-jobs). @behlendorf noted:

I can never remember the rabbit_workflow keyword. Could we get that added to the flux job info list of available keys?

[root@tuolumne1:nnf]# flux job info 
Usage: flux job info id key
some useful keys are:
J                    - signed jobspec
R                    - allocated resources
eventlog             - primary job eventlog
jobspec              - job specification
guest.exec.eventlog  - execution eventlog
guest.input          - job input log
guest.output         - job output log
Use flux job info -h to list available options

[grondo]

Well that list is just hardcoded in the flux job info command. I wonder, though, if instead the command with no args could list available keys that the current user can fetch? Maybe open an issue in flux-core if that idea is appealing.

[garlick]

So maybe do a recursive directory listing of the KVS job directory when no arguments are given.

I think the implementation would need to be a new RPC to job-info in order to support guests, so somewhat non-trivial but probably a good addition.

[wihobbs]

list available keys that the current user can fetch?

Will "added" keys have descriptions with them? (like where "R" says "allocated resources" next to it)

[grondo]

Will "added" keys have descriptions with them? (like where "R" says "allocated resources" next to it)

No. I was thinking they'd just be listed under "Other available keys:".

However, if we do want a list of keys + description, then flux job info would need to read some kind of config file (kind of like we implemented, but haven't really used for flux help) Then other framework projects that support extra keys could just drop the keys+description in a conf.d somewhere.

[chu11]

Could a --list option be a good compromise here? So we keep the default output with well known / defined keys. Tweak the output to say --list for full list and that option outputs the recursive true list?

[garlick]

The config file idea seems like it fulfills the requirements better and would be a whole lot easier to implement.

An alternative idea would be to have flux-coral2 provide a front end subcommand that does what flux job info does only for the rabbit keys. It wouldn't have to be very complex and could be written in python. A man page for the command could be a useful way to document the keys in more detail.

Thoughts? If palatable we can hot-potato this issue back to flux-coral2 :-)

[grondo]

Thoughts? If palatable we can hot-potato this issue back to flux-coral2 :-)

:joy: Actually not a bad idea at all! Better separation of concerns.

[wihobbs]

I like the config file idea! My only point was that those little blurbs can be really helpful. Particularly I would imagine for rabbit admins given the page @jameshcorbett linked to lists 6? that look kinda similar if you're flying fast