feat: Include images in call to LLM

[cecheta]

Required for #750

Purpose

• Pass retrieved images to LLM when advanced image processing is enabled

Does this introduce a breaking change?

• [ ] Yes
• [x] No

How to Test

• Run tests
• Deploy with advanced image processing enabled, upload images, ask questions based on images

[github-actions[bot]]

Coverage Report •

File	Stmts	Miss	Cover	Missing
code/backend/batch/utilities/tools
question_answer_tool.py	70	0	100%
TOTAL	2497	559	77%

Tests	Skipped	Failures	Errors	Time
277	0 :zzz:	0 :x:	0 :fire:	35.155s :stopwatch:

[cecheta]

Note that we ran into this issue where the LLM attempts to provide an answer from every image passed to the LLM, even if it's not relevant

This should be looked into during #964

[adamdougal]

Note that we ran into this issue where the LLM attempts to provide an answer from every image passed to the LLM, even if it's not relevant

This should be looked into during #964

Is this something to do with top_k? Or is this something we can tweak in the system prompt to say ignore anything not relevant?

[cecheta]

Note that we ran into this issue where the LLM attempts to provide an answer from every image passed to the LLM, even if it's not relevant This should be looked into during #964

Is this something to do with top_k? Or is this something we can tweak in the system prompt to say ignore anything not relevant?

I think it would need to be done in the prompt

[github-actions[bot]]

:tada: This PR is included in version 1.3.0 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket: