🚀 Feature: Caching for DocsGPT

[pabik]

🔖 Feature description

We need to implement caching for DocsGPT to improve performance and efficiency. If the same question is asked, using the same source and the same LLM, the result should be retrieved from the cache rather than triggering a new API call.

Redis is already configured and used for Celery tasks, so the cache system should leverage Redis for storing and retrieving these cached responses.

🎤 Why is this feature needed ?

This feature will improve the performance of DocsGPT by avoiding redundant API calls for identical requests. It will reduce response time for repeated queries, lower API costs, and improve user experience, especially for frequently asked questions.

When users ask similar questions repeatedly using the same data source, there's no need to re-run the same logic each time (at least for some time period). By introducing caching, we can streamline this process.

✌️ How do you aim to achieve this?

The implementation will involve:

1. Using Redis as the caching layer to store LLM responses, indexed by the combination of question, source, and LLM used.
2. Checking the cache before executing new LLM queries to see if a cached result is available.
3. Triggering from cache if an identical question is found in the cache, otherwise proceeding with the usual query process and then storing the result for future use.

This is a challenging task, and we'd love to collaborate on it. You can contribute directly in this issue or join the discussion in our Discord (collaborative-issues).

We also encourage splitting this issue into smaller, manageable tasks, but please link them back to this original issue for tracking purposes.

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

• [X] I checked and didn't find similar issue

Are you willing to submit PR?

None

[Arkyapatwa]

Hey @pabik I would like to work on this Issue.

[Arkyapatwa]

/assign

[fadingNA]

@pabik I have similar experience with the redis caching, if it ok can I work on this

Thanks.