search

kubeagi / arcadia

A diverse, simple, and secure all-in-one LLMOps platform
http://www.kubeagi.com/
Apache License 2.0
86 stars 23 forks source link

Shared GPU between models #450

Open bjwswang opened 10 months ago

nkwangleiGIT commented 10 months ago

This it to support the scenarios below?

  1. Run multiple model service on single GPU
  2. Isolation for single GPU's memory
bobsongplus commented 10 months ago

I could help with this issue. Could assign it to me?