search

gen-mind / cognix

Opensource private chatGPT for your organization knowledge
https://cognix.ch
Other
12 stars 4 forks source link

enable GPU on demand on rag #272

Open gsantopaolo opened 3 months ago

gsantopaolo commented 3 months ago

create a script that allows some pods to run on GPU on the rag cluster another script to stop the GPU instance

Make that K8 will do the GPU passthrough.

Service to run on GPU when enabled:

https://learn.microsoft.com/en-us/azure/aks/gpu-cluster?tabs=add-ubuntu-gpu-node-pool

noelhermans commented 3 months ago

Quota increased requested with Microsoft, but was denied for Eastindia region. @gsantopaolo requested quotas for EASTUS region, but it is not possbile to add a nodepool from another region to the Cognix AKS cluster

noelhermans commented 3 months ago

Waiting for Microsoft support to come back to us with list of GPU instances supported in East India region.