Deployments fail on GKE with kubernetes images > 1.26.3

GoogleCloudPlatform / llm-pipeline-examples

Apache License 2.0

105 stars 25 forks source link

Deployments fail on GKE with kubernetes images > 1.26.3 #48

Open Chris113113 opened 1 year ago

Chris113113 commented 1 year ago

There is a Cuda version mismatch present on at least 1.26.5 which results in both Deepspeed and Triton being unable to load models with the current inferencing images.