[core] Track user/service ID with AppWrapper

[MEllis-github]

Overview

For system monitoring and debugging, it is valuable to be able to obtain the service or user ID associated with the creation of an AppWrapper.

Questions

Are there plans to implement this? What is the right extension of the current design for doing so?

[asyoussef]

valuable, in my opinion, and we need it to be able to provide a user view of the dashboard. Whether thehe identity is injected in the AW or queried from the API server by the dashboard, can be debated. I would pick the easiest solution.

[asm582]

The quickest path is to use helm and add a new field to labels as shown below:

Run below command to get openshift userid

user_id = oc whoami

add the new field to metadata block of appwrapper

metadata:
  name: aw-schd-spec-with-timeout-1
  namespace: default
  #new addition - starts
  labels:
  userid: user_id #env var that has output of  oc whoami command
  #new addition - ends

[MEllis-github]

Is this proposal for MCAD's helm charts or for MCAD-external users/tools?

[asm582]

This proposal is for the helm tool that users use to submit app wrappers in the foundational model stack. No changes are needed on the MCAD side for now

[MEllis-github]

Multiple methods are used for submitting appwrappers at this point in time e.g. the codeflare CLI tool, torchX+MCAD, custom helm charts and scripts, in addition to direct manifest editing and submission. In other words, the method is not standardized even for the foundation model stack, so tracking on the submission tool side will have limited coverage.

[asm582]

@astefanutti @anishasthana @dimakis your thoughts?

[astefanutti]

This is generally achieved by multi-tenancy support, with namespace as the tenancy unit. Given the distributed and asynchronous nature of Kubernetes API / controllers, it's often intractable to implement an ad-hoc propagation of the end-user identity.

Practically for the dashboard case, the UI would display to the end-user the list of AppWrapper she/he has permission to, based on Kubernetes RBAC and the namespaces / tenants that end-user can access.