search

databendlabs-x / askbend

AskBend: SQL-based Knowledge Base Search and Completion using Databend
https://ask.databend.rs
Apache License 2.0
191 stars 15 forks source link

Code: 1001, Text = openai embedding request error #86

Open elvin-du opened 1 year ago

elvin-du commented 1 year ago

when I run ./target/release/askbend -c askbend.toml --rebuild cmd , I got the error as blow. how to fix this? 

[2023-10-16T02:07:41Z INFO ] embedding 605/1936,  time: 513.095121ms
Error: APIError: ResponseError with 1001: openai embedding request error:Internal. Code: 1001, Text = openai embedding request error: ApiError("{\"error\":{\"message\":\"This model's maximum context length is 8191 tokens, however you requested 8916 tokens (8916 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.\",\"type\":\"invalid_request_error\",\"param\":null,\"code\":null}}").
BohuTANG commented 1 year ago

context length is 8191 tokens please split your content to more small piece.

elvin-du commented 1 year ago

content means the content of one file? I just put 390 files on the path which set in config file.

BohuTANG commented 1 year ago

One of your file length is larger than 8191(prompts length + content length = 8916). We should split this file into two parts, make sure not near 8191.