Closed ZeroExistence closed 1 year ago
Volcano admission is a simple admission controller and webhook certificate generation component, which is very simple in function. It's great if you'd like to create a doc for it. What information do you want to know?
Hi @hwdef, thank you for looking into my inquiry! This would be the possible scenario. Assuming I currently use these settings for Volcano admission.
apiVersion: v1
data:
volcano-admission.conf: |
#resourceGroups:
#- resourceGroup: management # set the resource group name
# object:
# key: namespace # set the field and the value to be matched
# value:
# - mng-ns-1
# schedulerName: default-scheduler # set the scheduler for patching
# tolerations: # set the tolerations for patching
# - effect: NoSchedule
# key: taint
# operator: Exists
# labels:
# volcano.sh/nodetype: management # set the nodeSelector for patching
#- resourceGroup: cpu
# object:
# key: annotation
# value:
# - "volcano.sh/resource-group: cpu"
# schedulerName: volcano
# labels:
# volcano.sh/nodetype: cpu
#- resourceGroup: gpu # if the object is unsetted, default is: the key is annotation,
# schedulerName: volcano # the annotation key is fixed and is "volcano.sh/resource-group", The corresponding value is the resourceGroup field
# labels:
# volcano.sh/nodetype: gpu
resourceGroups:
- resourceGroup: training-pool-volcano
object:
key: annotation
value:
- "user: someone"
schedulerName: volcano
labels:
user-label: someone
- resourceGroup: training-pool-job
object:
key: annotation
value:
- "name: test-job"
schedulerName: volcano
labels:
name-label: test-job
kind: ConfigMap
metadata:
annotations:
meta.helm.sh/release-name: volcano
meta.helm.sh/release-namespace: volcano-system
labels:
app.kubernetes.io/managed-by: Helm
name: volcano-admission-configmap
namespace: volcano-system
I would like to have some more information about these items if possible.
training-pool-volcano
resourceGroup is mutating the pod's nodeSelector. If the pod has user: someone
annotation, additional user-label: someone
nodeSelector will be added to the pod. Is this correct?training-pool-volcano
and training-pool-job
resourceGroup, it looks to me that only the first resourceGroup mutation will be applied? Is this right?nodeSelector
setup? I mean, is it other fields that can be mutated in pod's manifest?jobs.batch.volcano.sh/v1alpha1
for now, right? If I want to mutate normal Kubernetes jobs, I probably need to edit or just create new webhook handler for it inside the Volcano admission handler? Or just create a whole new mutation/validation controller.resourceGroup
to apply for jobs, and not pods? There might be a case that both job and pod have same annotations and labels, so the resourceGroup
might be applied in both job and pod. queue
. I assume mutation would not be applicable that much for queue, since there would less scenario to actively create new queues.podgroup
as well?If I have some more details on these items, I can create a simple graph to have a visual documentation on how it works.
Thank you very much!
I just realized that pod resources are the only configurable component in Volcano webhook, upon checking the reference to pkg/webhooks/config/config.go
. I had an initial assumption that most of the resources are configurable. I assume the other webhook handlers (job, podgroup, queue) are just used by Volcano internally then.
I guess having a dedicated page for Volcano admission/webhook features and limitations would simplify it 😄
I will start checking and making documentation draft once the items above and the information here below is confirmed. Thank you very much!
Hello 👋 Looks like there was no activity on this issue for last 90 days. Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗 If there will be no activity for 60 days, this issue will be closed (we can always reopen an issue if we need!).
Closing for now as there was no activity for last 60 days after marked as stale, let us know if you need this to be reopened! 🤗
What would you like to be added:
I would like to check if there are existing documentation on the workflow and process of Volcano admission/webhook component. Currently, I only had some time to explore the pod mutating and validation feature of Volcano. Upon checking in both the
docs
directory and the Volcano's official website, there are not much information about the Volcano webhook handler.Why is this needed:
This is to provide more information on how to customize Volcano webhook and utilize them for extending Volcano capabilities.
If I got some information on what does the webhooks do in the Volcano, I don't mind creating a
.md
documentation for this. Thanks!