search

mckaywrigley / repo-chat

Use AI to ask questions about any GitHub repo.
MIT License
604 stars 108 forks source link

Max token limit #5

Open hubsmoke opened 1 year ago

hubsmoke commented 1 year ago

Was able to generate embeddings, when asking a simple question "what does this repo do" I get

2023-04-26 14:24:46,237:INFO - error_code=context_length_exceeded error_message="This model's maximum context length is 4097 tokens. However, your messages resulted in 72361 tokens. Please reduce the length of the messages." error_param=messages error_type=invalid_request_error message='OpenAI API error received' stream_error=False

Is there any guidance on how to tune this to fall within the limits?

I should also note on the embedding step I receive several warnings


2023-04-26 14:30:53,281:WARNING - Created a chunk of size 2374, which is longer than the specified 2000
2023-04-26 14:30:53,282:WARNING - Created a chunk of size 2182, which is longer than the specified 2000
2023-04-26 14:30:53,284:WARNING - Created a chunk of size 2481, which is longer than the specified 2000
2023-04-26 14:30:53,285:WARNING - Created a chunk of size 2261, which is longer than the specified 2000
2023-04-26 14:30:53,289:WARNING - Created a chunk of size 2182, which is longer than the specified 2000
2023-04-26 14:30:53,290:WARNING - Created a chunk of size 9919, which is longer than the specified 2000
hubsmoke commented 1 year ago

I was able to work around this by adding more exclusions to the embed.py file, to remove unnecessary files

saydus commented 1 year ago

Can you share how you overcame this?

hubsmoke commented 1 year ago

1) Remove all unnecessary files from the folder copy of the repo you are analyzing. Alternatively, add all unnecessary files to the exclusion list in embed.py (you'll see a few in there pre-filled like package-lock.json). 2) Clear vector db 3) Re-run embeddings script 3) Ask questions

It took me a few tries but I got it answering basic questions. However, I still had this problem reoccur for certain questions.

I believe this has to do with returning too much text from the vectordb, because perhaps the indices are not split up optimally for my data, I think this parameter is tuneable in embed.py. I haven't messed with it further, but will check it out again later