Genv is an open-source environment and cluster management system for GPUs.
Genv lets you easily control, configure, monitor and enforce the GPU resources that you are using in a GPU machine or cluster.
It is intendend to ease up the process of GPU allocation for data scientists without code changes 💪🏻
Check out the Genv documentation site for more details and the website for a higher-level overview of all features.
This project was highly inspired by pyenv and other version, package and environment management software like Conda, nvm, rbenv.
Plus, it's 100% free and gets installed before you can say Jack Robinson.
Ready to create an LLM playground for yourself and your teammates?
Genv integrates with Ollama for managing Large Language Models (LLMs). This allows users to efficiently run, manage, and utilize LLMs on GPUs within their clusters.
$ genv remote -H gpu-server-1, gpu-server-2 llm serve llama2 --gpus 1
Check out our documentation for more information.
Make sure that you are running on a GPU machine:
$ nvidia-smi
Tue Apr 4 11:17:31 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.03 Driver Version: 470.161.03 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
...
pip install genv
conda install -c conda-forge genv
$ genv status
Environment is not active
$ genv activate –-name my-env —-gpus 1
(genv:my-env)$ genv config gpu-memory 4g
(genv:my-env)$ genv status
Environment is active (22716)
Attached to GPUs at indices 0
Configuration Name: my-env Device count: 1 GPU memory capacity: 4g
4. Start working on your project!
## :scroll: Documentation
Check out the Genv [documentation site](https://docs.genv.dev) for more details.
## :dizzy: Simple Integration & Usage with your fav IDE
Integration with VSCode [(Take me to the installation guide!)](https://github.com/run-ai/vscode-genv) |
:-------------------------:|
<img src="https://github.com/run-ai/genv/raw/main/images/overview.gif" width="800" alt="genv vscode"/> |
<br />
Integration with JupyterLab [(Take me to the installation guide!)](https://github.com/run-ai/jupyterlab_genv) |
:-------------------------:|
<img src="https://github.com/run-ai/genv/raw/main/images/overview_jupyterlab.gif" width="800" alt="genv jupyterlab"/> |
A PyCharm integration is also in our roadmap so stay tuned!
## 🏃🏻 Join us in the AI Infrastructure Club
[<img src="https://img.shields.io/badge/Discord-Join%20the%20community!-7289da?style=for-the-badge&logo=discord&logoColor=7289da" height="30" />](https://discord.gg/zN3Q9pQAuT)
We love connecting with our community, discussing best practices, discovering new tools, and exchanging ideas with makers about anything around making & AI infrastructure. So we created a space for all these conversations. Join our Discord server for:
- Genv Installation and setup support as well as best practice tips and tricks directly for your use-case
- Discussing possible features for Genv (we prioritize your requests)
- Chatting with other makers about their projects & picking their brain up when you need help
- Monthly Beers with Engineers sessions with amazing guests from the research and industry ([Link to previous session recordings](https://www.youtube.com/@runai_/search?query=beers%20with%20engineers))
## License
The Genv software is Copyright 2022 [Run.ai Labs, Ltd.].
The software is licensed by Run.ai under the AGPLv3 license.
Please note that Run.ai’s intention in licensing the software are that the obligations of licensee pursuant to the AGPLv3 license should be interpreted broadly.
For example, Run.ai’s intention is that the terms “work based on the Program” in Section 0 of the AGPLv3 license, and “Corresponding Source” in Section 1 of the AGPLv3 license, should be interpreted as broadly as possible to the extent permitted under applicable law.