[Admission] Documentations about Volcano Admission and Webhooks

[ZeroExistence]

What would you like to be added:

I would like to check if there are existing documentation on the workflow and process of Volcano admission/webhook component. Currently, I only had some time to explore the pod mutating and validation feature of Volcano. Upon checking in both the docs directory and the Volcano's official website, there are not much information about the Volcano webhook handler.

Why is this needed:

This is to provide more information on how to customize Volcano webhook and utilize them for extending Volcano capabilities.

If I got some information on what does the webhooks do in the Volcano, I don't mind creating a .md documentation for this. Thanks!

[hwdef]

Volcano admission is a simple admission controller and webhook certificate generation component, which is very simple in function. It's great if you'd like to create a doc for it. What information do you want to know?

[ZeroExistence]

Hi @hwdef, thank you for looking into my inquiry! This would be the possible scenario. Assuming I currently use these settings for Volcano admission.

apiVersion: v1
data:
  volcano-admission.conf: |
    #resourceGroups:
    #- resourceGroup: management                    # set the resource group name
    #  object:
    #    key: namespace                             # set the field and the value to be matched
    #    value:
    #    - mng-ns-1
    #  schedulerName: default-scheduler             # set the scheduler for patching
    #  tolerations:                                 # set the tolerations for patching
    #  - effect: NoSchedule
    #    key: taint
    #    operator: Exists
    #  labels:
    #    volcano.sh/nodetype: management           # set the nodeSelector for patching
    #- resourceGroup: cpu
    #  object:
    #    key: annotation
    #    value:
    #    - "volcano.sh/resource-group: cpu"
    #  schedulerName: volcano
    #  labels:
    #    volcano.sh/nodetype: cpu
    #- resourceGroup: gpu                          # if the object is unsetted, default is:  the key is annotation,
    #  schedulerName: volcano                      # the annotation key is fixed and is "volcano.sh/resource-group", The corresponding value is the resourceGroup field
    #  labels:
    #    volcano.sh/nodetype: gpu
    resourceGroups:
    - resourceGroup: training-pool-volcano
      object:
        key: annotation
        value:
        - "user: someone"
      schedulerName: volcano
      labels:
        user-label: someone
    - resourceGroup: training-pool-job
      object:
        key: annotation
        value:
        - "name: test-job"
      schedulerName: volcano
      labels:
        name-label: test-job
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: volcano
    meta.helm.sh/release-namespace: volcano-system
  labels:
    app.kubernetes.io/managed-by: Helm
  name: volcano-admission-configmap
  namespace: volcano-system

I would like to have some more information about these items if possible.

• Pod mutation
  • Based on my configuration above, it seems that the training-pool-volcano resourceGroup is mutating the pod's nodeSelector. If the pod has user: someone annotation, additional user-label: someone nodeSelector will be added to the pod. Is this correct?
  • If the pod matches both the training-pool-volcano and training-pool-job resourceGroup, it looks to me that only the first resourceGroup mutation will be applied? Is this right?
  • Is pod mutation only geared towards automated nodeSelector setup? I mean, is it other fields that can be mutated in pod's manifest?
• Job mutation
  • It seems that job mutation is only available for jobs.batch.volcano.sh/v1alpha1 for now, right? If I want to mutate normal Kubernetes jobs, I probably need to edit or just create new webhook handler for it inside the Volcano admission handler? Or just create a whole new mutation/validation controller.
  • I assume job mutation has same filtering with pod (ex. filtered with annotation or namespace). But what can I mutate on this case? Would adding same labels would just add new labels to job?
  • How can I specify if I only want a resourceGroup to apply for jobs, and not pods? There might be a case that both job and pod have same annotations and labels, so the resourceGroup might be applied in both job and pod.
• Queue mutation
  • What can I mutate in queue. I assume mutation would not be applicable that much for queue, since there would less scenario to actively create new queues.
• Podgroup mutation
  • What can I mutate with podgroup as well?

If I have some more details on these items, I can create a simple graph to have a visual documentation on how it works.

Thank you very much!

[ZeroExistence]

I just realized that pod resources are the only configurable component in Volcano webhook, upon checking the reference to pkg/webhooks/config/config.go. I had an initial assumption that most of the resources are configurable. I assume the other webhook handlers (job, podgroup, queue) are just used by Volcano internally then.

I guess having a dedicated page for Volcano admission/webhook features and limitations would simplify it 😄

I will start checking and making documentation draft once the items above and the information here below is confirmed. Thank you very much!

[stale[bot]]

Hello 👋 Looks like there was no activity on this issue for last 90 days. Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity for 60 days, this issue will be closed (we can always reopen an issue if we need!).

[stale[bot]]

Closing for now as there was no activity for last 60 days after marked as stale, let us know if you need this to be reopened! 🤗