Closed milroy closed 2 years ago
I added a getter class for the templates since I realized it's pretty useful to be able to print the available templates and their parameter overrides. I also restructured the class inheritance for simplicity and initialization performance.
The latest force pushes introduce functionality that implements oc get
for OpenShift object types.
They also include mini submit
and mini cancel
which behave like flux mini submit
in that they generate a skeleton jobspec with the specified OpenShift template and parameter overrides.
Note that I can add a new commit to change the file name to flux-oc.py
, which will make the CLI invocation flux oc
per the discussion several weeks ago with @dongahn.
Thank you for the thorough and helpful reviews @SteVwonder, @dongahn, and @grondo! I think I've addressed all but two issues so you can give this PR a final pass.
Regarding the other two issues, perhaps we can discuss during coffee time?
Thank you to everyone for your thorough and helpful reviews! This contribution is much more fully-featured and hardened than it would have been otherwise.
This PR adds all the functionality needed for the Unified Interface project. It features three classes of functionality.
First, the functionality of mapping a Flux jobspec to an OpenShift template. The library takes a Flux jobspec and matches the jobspec command to a corresponding OpenShift template name. It then matches the template parameter overrides specified in the jobspec's system attributes to the overrides available in the selected template. The library returns a service, which is the selected template with the dictionary of parameter overrides.
Second, this PR adds the
flux-kube.so
jobtap plugin and the associated machinery to build and install it. The jobtap plugin is based on alloc-bypass.so (flux-core90461710f1f09ed48cc7586abff5cb4fee6dc63d
). Upon ingest, the plugin looks for attributes.system.flux-kube set in the jobspec. If it's present, the plugin sets the "alloc-bypass" flag, passes the jobspec to flux-kube.py via RPC, and unpacks the response. Then the plugin generates a dummy R (since the resources will reside on OpenShift) and commits it to the KVS. Finally, the plugin emits an "alloc" event and then submits the jobspec via asynchronous RPC to theflux kube
daemon for creation of OpenShift objects. Cancellation works analogously, sending the stored jobspec to theflux kube
daemon via an asynchronous RPC.Third, this PR adds the fully-featured
flux kube
CLI and daemon. The CLI featurestranslate
,submit
,cancel
, anddaemonize
subcommands. Translate uses kube_translate.py to translate a Flux jobspec into an OpenShift template with parameter overrides read from system.attributes. Submit takes either a jobspec or string input and creates the corresponding OpenShift objects. Unfortunately, the OpenShift APIs do not have the required functionality (present in theoc
binary) to convert templates to object YAMLs. Therefore, we must calloc process
and pipe it tooc create
via Python subprocess calls. Cancel works similarly tosubmit
, but deletes the OpenShift objects.Daemonize registers a Flux service that creates a watcher callback for jobspecs sent via RPC from the
flux-kube.so
jobtap. It creates the OpenShift objects and returns a success/failure value to the jobtap. The daemon is started as a job at rc1 viaflux mini submit
so that it can be cleaned up when Flux exits.Much of the functionality of this PR requires OpenShift templates. The functionality will be extended to interface with base K8s CRDs (which can replicate the template functionality) in the future.