There is contextually relevant information in different modalities that can enrich the question space.
Implementation guidance
This would affect all files within the services directory, as well as their associated routers and models. We need to add the option for a multimodal model, as well as probably connecting to a local instance. This should be an optional install in the virtual environment.
What
Include ColPali for multimodal extraction from PDFs, so questions can be asked of more than just text.
https://github.com/AnswerDotAI/byaldi
Why
There is contextually relevant information in different modalities that can enrich the question space.
Implementation guidance
This would affect all files within the services directory, as well as their associated routers and models. We need to add the option for a multimodal model, as well as probably connecting to a local instance. This should be an optional install in the virtual environment.