Closed ahg-g closed 1 week ago
+1, this will be very helpful!
Can I try it? I will spend some time on it :)
/assign
friendly ping :) @ahg-g @liurupeng This is an example I got. I want to know one thing. What we need to do is inject the spec.leaderWorkerTemplate.size field into the container, right?
apiVersion: leaderworkerset.x-k8s.io/v1
kind: LeaderWorkerSet
metadata:
labels:
app.kubernetes.io/name: leaderworkerset
app.kubernetes.io/instance: leaderworkerset-multi-template
app.kubernetes.io/part-of: lws
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: lws
name: leaderworkerset-multi-template
spec:
replicas: 3
leaderWorkerTemplate:
leaderTemplate:
spec:
containers:
- name: nginx2
image: nginx:1.14.2
resources:
limits:
cpu: "100m"
requests:
cpu: "50m"
ports:
- containerPort: 8080
size: 4
workerTemplate:
spec:
containers:
- name: nginx
image: nginx:1.14.2
resources:
limits:
cpu: "100m"
requests:
cpu: "50m"
ports:
- containerPort: 8080
If so, I plan to use this annotationKey directly to implement this feature
// Size will be added to leader pods as an annotation which corresponds to
// LeaderWorkerSet.Spec.LeaderWorkerTemplate.Size.
SizeAnnotationKey string = "leaderworkerset.sigs.k8s.io/size"
thanks for picking it up @googs1025
It's done now. If we have any questions, feel free to reopen. :)
/close
@googs1025: Closing this issue.
What would you like to be added:
Add the group size as an env var
Why is this needed:
In most cases for multi-host inference, the size is needed, like in vllm.
Suggest to use LWS_GROUP_SIZE as the env var name.
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.