search

theodo-group / LLPhant

LLPhant - A comprehensive PHP Generative AI Framework using OpenAI GPT 4. Inspired by Langchain
MIT License
761 stars 76 forks source link

[Feature] Re-ranking and prompt compression #180

Closed synio-wesley closed 1 month ago

synio-wesley commented 1 month ago

What I want to achieve basically is re-ranking and prompt compression, before adding the retrieved docs to the context.

I read that it could drastically improve RAG performance. I think right now the QuestionAnswering class does not let us hook into modifying the list of retrieved docs.

To at least support a third party to do this, we could maybe add something similar to the QueryTransformer with the default IdentityTransformer which can be passed when constructing a QuestionAnswering object, but instead it would be something like RetrievedDocumentsTransformer with a default (or null) transformer that does nothing.

Because I think this is not supported yet without modifying LLPhant?

Then later we can also add some RetrievedDocumentsTransformer classes like one for re-ranking and one for prompt compression. And then we can also create a multi-transformer that allows us to define multiple other transformers, so we can combine multiple transformers and run them one by one.

ClicShopping commented 1 month ago

Hello, I open a post about that . https://github.com/theodo-group/LLPhant/issues/173 but I decided to close. After conversation with some people, this approach is not relevant and we can loose some information about the context. After we can take in consideration the cost of llm will decrease. Look what s happen with gpt4-o mini. In this case not sure it's really important to include this kind of feature. That's my opinion.

synio-wesley commented 1 month ago

@ClicShopping I'm not talking about base64 compression, but rather optimizing the prompt somehow by re-writing it (for example using an LLM as well). And I'm also more interested in re-ranking, which has proven more successful.

In any case, I think this is a valid feature request to want to hook a custom transformer right after documents have been retrieved, so we can re-rank them, or do anything else with them to optimize it, which can be dependent on your specific requirements.

ClicShopping commented 1 month ago

ok understood

MaximeThoonsen commented 1 month ago

Hey @synio-wesley , In langchain we do a custom Chain for that. SO we could create another QA class with what you want to do and keep this one straightforward. What do you think?

f-lombardo commented 1 month ago

@MaximeThoonsen I'm not sure if creating a QA class could simplify things. I think that we should keep both updated when we'll change one of them. I think that @synio-wesley proposal of adding a RetrievedDocumentsTransformer seems fair enough. I'll try to propose an implementation in the next few days

synio-wesley commented 1 month ago

I do have some code that I am using myself right now. I could maybe share it as an example but the code needs some tweaking and tests before it is ready.. I'm on holiday right now for 2 weeks so will have to see when I have some time.

MaximeThoonsen commented 1 month ago

merged. All good for you @synio-wesley ?