[DMP 2024]: Fractional GPU Utilisation

[singhalkarun]

Ticket Contents

Description

Create a linux package with an exposed interface that allows the utilisation of a fraction of a GPU, similar to how cgroups works. This package will enable users to efficiently allocate and manage GPU resources for their applications.

Reference 1: https://docs.run.ai/v2.15/Researcher/Walkthroughs/walkthrough-fractions/ Reference 2: https://github.com/AliyunContainerService/gpushare-scheduler-extender

Goals & Mid-Point Milestone

Mid Point Milestone

Linux Package with following APIs/Functionalities :

1. Reserve GPU for a Processes
2. Get Free GPU
3. Get a list of Current Processes

Goals

• Linux Package: -Develop a Linux package with an exposed interface for managing GPU resources.
  • Implement functionality to allocate and control a fraction of a GPU for application usage.
• Plugin for Docker Compose:
  • Create a plugin for Docker Compose that integrates with the Linux package.
  • Enable users to specify GPU resource requirements for their Docker Compose services.
• Plugin for Kubernetes (K8S):
  • Develop a plugin for Kubernetes that utilizes the Linux package.
  • Allow users to specify GPU resource limits and requests in Kubernetes pods.

Setup/Installation

No response

Expected Outcome

• A Linux package that provides an APIs for managing GPU resources.
• A Docker Compose plugin enabling users to specify GPU resource requirements for their services.
• A Kubernetes plugin allowing users to specify GPU resource limits and requests for pods.

Acceptance Criteria

• The Linux package interface allows users to allocate and control a fraction of a GPU.
• The Docker Compose plugin seamlessly integrates with the Linux package, enabling GPU resource specification.
• The Kubernetes plugin effectively utilizes the Linux package for managing GPU resources in pods.
• Extensive testing is conducted to ensure compatibility and reliability across different environments.

Implementation Details

• Develop the framework-agnostic Linux package with a clear and intuitive interface for GPU resource management.
• Implement the Docker Compose plugin to communicate with the Linux package and translate resource specifications.
• Create the Kubernetes plugin to interact with the Linux package and enforce GPU resource limits and requests in pods.

Mockups/Wireframes

No response

Product Name

BharatSahAIyak

Organisation Name

SamagraX

Domain

Open Source Library

Tech Skills Needed

Docker Linux - Internals, Networking GPU (good starter)

Mentor(s)

@singhalkarun

Category

AI

[RohanHBTU]

hi @singhalkarun, I already have WSL in my machine and have past experience with docker too. I am highly obsessed with GPU and Linux. Can you give me guidance regarding what approach should be considered, so that I can contemplate about it further.

Thank you for your time and consideration.

[AbhimanyuSamagra]

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

[Bre-31]

hello @singhalkarun , i will be actively working on this project.

[H4R5H1T-007]

I have found out genv library from run.ai maybe we can use this but the problem is they are working on integration with kubernetes and it's not complete.

[Saswatsusmoy]

@singhalkarun @AbhimanyuSamagra

I have few questions regarding the project and how I should approach it

• What specific APIs and functions should the Linux package expose for managing GPU resources? Should it follow any existing standards or conventions?

• How can the Linux package ensure compatibility with different GPU models and drivers? Are there any specific considerations for NVIDIA GPUs using CUDA or AMD GPUs using ROCm?