Vertex AI API needs to be updated. - Githubissues

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs

Apache License 2.0

590 stars 93 forks source link

Vertex AI API needs to be updated. #48

Open Durga2Dash opened 4 months ago

Durga2Dash commented 4 months ago

Team,

Looks like the Google Cloud Vertex AI API used might be outdated.

Below is the current api in code https://{region}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{region}/endpoints/{endpoint_id}:predict

Below is the latest API that works [have tested by updating code on my end] https://{region}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{region}/publishers/google/models/{model}:predict

File: src/llmperf/ray_clients/vertexai_client.py

Thanks