search

openshift / instaslice-operator

InstaSlice Operator facilitates slicing of accelerators using stable APIs
Apache License 2.0
15 stars 12 forks source link

Add env variable check in webhook at submit time #186

Open asm582 opened 1 month ago

asm582 commented 1 month ago

In the current setup, NVIDIA_VISIBLE_DEVICES env variable is added to configmap so that we pin the pod to a MIG slice. A user pod could have this variable set in the pod at submit time which will provide container access to a slice not chosen by InstaSlice and in the worst case access to all the GPUs on the node. We should modify the webhook to reject such pods at submit time.

harche commented 1 month ago

Also, we can consider modifying the webhook to intercept not just pod creation but pod update to make sure users don't intentionally set NVIDIA_VISIBLE_DEVICES to 0.