Open BaileySimrell opened 4 months ago
I have made changes to the app you can fork it here and check it https://github.com/Ghastly2001/openai-assistants-quickstart
@Ghastly2001 PRs welcome!
This should be higher up. I left a few tabs open and burned through over 70gb of bandwidth in only 2 days.
Issue Overview
I am experiencing an issue where only a subset (20 out of 300) of the files in my vectorstore is visible in the UI of the file viewer component. Despite attempts to resolve this by fetching additional data and handling pagination, I keep encountering rate limit errors that hinder further data fetching.
Steps to Reproduce
Expected Behavior
The file viewer should correctly paginate through all files in the vectorstore, displaying all (n) files without hitting rate limits prematurely.
Actual Behavior
Only 20 files are displayed repeatedly, and attempts to fetch more files frequently hit the rate limit, even with extremely conservative fetch intervals (e.g., every 30 seconds).
Possible Solutions or Suggestions
Additional Context
Here's the logic I've tried implementing to handle fetching, with adjustments for rate limits:
Any insights or suggestions on how to properly paginate and handle rate limits in this scenario would be greatly appreciated!