Closed ccreutzi closed 3 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 97.11%. Comparing base (
fce8bb8
) to head (b5ad5d5
). Report is 2 commits behind head on main.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
I am happy with this change. However, if we see that moondream makes the tests flaky, we could try out llava:7b which is a bit larger and might provide more robust answers for the test purposes.
We just moved away from bakllava:7b, which is just as big as llava:7b, because of the download size.
This is the only test point using a vision model in Ollama, at least so far.
The moondream model is unreliable in reporting peppers or even reporting anything at all, notice the empty responses in several of these calls:
There have also been multiple CI failures like this one:
Since we are not interested in testing the model, but we do want to run an end-to-end test to ensure that we pass images in the data format required by Ollama, generate multiple responses and make sure that at least one of them mentions
"pepper"
or"vegetable"
.