databricks-industry-solutions / hls-llm-doc-qa

Build a question answering system based on a given collection of documents with open-source LLMs
Other
47 stars 23 forks source link

400 Bad request error on create_endpoint() #9

Closed CarlosLabrado closed 1 year ago

CarlosLabrado commented 1 year ago

I am getting this error:

400 Client Error: Bad Request for url: https://<my_endpoint>.azuredatabricks.net/api/2.0/serving-endpoints When executing this line:

create_endpoint(endpoint_name, served_models)

I think is because GPU Medium is not an option when registering an endpoint.

Above that line we can see that we are declaring the workload_type:

served_models = [
    {
      "name": model_name,
      "model_name": model_name,
      "model_version": model_version,
      "workload_size": "Small",
      "workload_type": "GPU_MEDIUM",
      "scale_to_zero_enabled": False
    }
]

I tried the registry in Isolation and getting this error:

{
  "error_code": "INVALID_PARAMETER_VALUE",
    "message": "Workload type 'GPU_MEDIUM' with size 'Small' is not supported. Please choose a node type from "
}
CarlosLabrado commented 1 year ago

I know why, is because GPU Medium doesn't exist in the Azure Cloud.