Allow GPU multi-provisioning acording to the amount of free memory.

run-ai / genv

GPU environment and cluster management with LLM support

https://www.genv.dev

GNU Affero General Public License v3.0

447 stars 20 forks source link

Closed davidLif closed 1 year ago

davidLif commented 1 year ago

Allow GPU multi-provisioning acording to the amount of free memory. Environment without a gpu memory configuration - takes a full gpu.

Known issue - not device refresh