databricks-demos / dbdemos

Demos to implement your Databricks Lakehouse
Other
255 stars 80 forks source link

VectorIndexIssue with 1/ Data preparation for LLM Chatbot RAG #124

Open SiddyP opened 1 month ago

SiddyP commented 1 month ago

Running the tutorial as-is doesn't seem to work properly. Attempting to create vector index with below name fails index name: user_siddy_persson.rag_chatbot.databricks_documentation_vs_index

DLT-error: Failed to resolve flow: '__online_index_view'.

With the underlying error:

java.lang.Exception: Error: Response Code: 400, Response: {"error_code":"INVALID_PARAMETER_VALUE","message":"Failed to call Model Serving endpoint: bge-small-en."}

Stacktrace in the notebook:

Exception: Response content b'{"error_code":"INVALID_PARAMETER_VALUE","message":"Invalid index name. Must specify the full index name <catalog>.<schema>.<table>. Only alphanumerics and underscores are allowed."}', status_code 400

Changing the index name appears so solve the issue, e.g.: documentationindex works. Meaning my fully qualified index resource in UC becomes user_siddy_persson.rag_chatbot.documentationindex

Also, I think it'd be useful to clarify even further that the embedding endpoint is expected to exist in the workspace (and can't start with databricks_)