few small fixes for http tgi route

GoogleCloudPlatform / ai-on-gke

AI on GKE is a collection of examples, best-practices, and prebuilt solutions to help build, deploy, and scale AI Platforms on Google Kubernetes Engine

Apache License 2.0

220 stars 165 forks source link

few small fixes for http tgi route #675

Closed annapendleton closed 4 months ago

annapendleton commented 4 months ago

A few additional fixes required for http route

Tested end to end, validated locust comes up, and successful response.

Updates:

remove single ' in run.sh
remove self. references to methods that no longer belong to http user class
update recommended defaults in sample tfvars file to work for tgi http