A topology and heterogeneous resource aware scheduler for fractional GPU allocation in Kubernetes cluster
KubeShare 2.0 is designed in the way of the scheduling framework.
Note that KubeShare 1.0 is deprecated. Refer to the KubeShare 1.0 branch for the old version.
NVIDIA_VISIBLE_DEVICES
to control which GPUs will be made accessible inside the container.Because floating point custom device requests is forbidden by K8s, we move GPU resource usage definitions to Labels.
sharedgpu/gpu_request
(required if allocating GPU): guaranteed GPU usage of Pod, gpu_request <= "1.0".sharedgpu/gpu_limit
(required if allocating GPU): maximum extra usage if GPU still has free resources, gpu_request <= gpu_limit <= "1.0".sharedgpu/gpu_mem
(optional): maximum GPU memory usage of Pod, in bytes. The default value depends on gpu_request
sharedgpu/priority
(optional): pod priority 0~100. The default value is 0.
sharedgpu/pod_group_name
(optional): the name of pod group for a coschedulingsharedgpu/group_headcount
(optional): the total number of pods in same groupsharedgpu/group_threshold
(optional): the minimum proportion of pods to be scheduled together in a pod group.apiVersion: v1
kind: Pod
metadata:
name: mnist
labels:
"sharedgpu/gpu_request": "0.5"
"sharedgpu/gpu_limit": "1.0"
"sharedgpu/gpu_model": "NVIDIA-GeForce-GTX-1080"
spec:
schedulerName: kubeshare-scheduler
restartPolicy: Never
containers:
- name: pytorch
image: riyazhu/mnist:test
command: ["sh", "-c", "sleep infinity"]
imagePullPolicy: Always #IfNotPresent
apiVersion: batch/v1
kind: Job
metadata:
name: lstm-g
labels:
app: lstm-g
spec:
completions: 5
parallelism: 5
template:
metadata:
name: lstm-o
labels:
"sharedgpu/gpu_request": "0.5"
"sharedgpu/gpu_limit": "1.0"
"sharedgpu/group_name": "a"
"sharedgpu/group_headcount": "5"
"sharedgpu/group_threshold": "0.2"
"sharedgpu/priority": "100"
spec:
schedulerName: kubeshare-scheduler
restartPolicy: Never
containers:
- name: pytorch
image: riyazhu/lstm-wiki2:test
# command: ["sh", "-c", "sleep infinity"]
imagePullPolicy: IfNotPresent
volumeMounts:
- name: datasets
mountPath: "/datasets/"
volumes:
- name: datasets
hostPath:
path: "/home/riya/experiment/datasets/"
git clone https://github.com/NTHU-LSALAB/KubeShare.git
cd KubeShare
make
make build-image
make push-image
CONTAINER_PREFIX
, CONTAINER_NAME
, CONTAINER_VERSION
Please refer to Gemini.
Any issues please open a GitHub issue, thanks.