PeterJT
commented
2 weeks ago As an FYI , using OpenAI and the gpt-4o-mini model, I had no issues with that image / recipe.
Closed death2all110 closed 2 weeks ago
PeterJT
commented
2 weeks ago As an FYI , using OpenAI and the gpt-4o-mini model, I had no issues with that image / recipe.
death2all110
commented
2 weeks ago Thanks for confirming. I tested calling the API with cURL and that seemed to work with another image, and the working image from cURL does not work in Mealie either. So It does not appear to be an ollama issue, but the way Mealie is calling the API.
Gangoke
commented
2 weeks ago I'm also getting Error 400 Bad Request trying to use ollama (llava) for Create from Image. Parser is working tho so i know Mealie can talk to Ollama. Like you said, must be the way Mealie is calling the API for images.
death2all110
commented
2 weeks ago @michael-genson (Sorry to tag you directly, but you were the main guy behind the OpenAI implementation, and I figured since I saved you all a bunch of debug and troubleshooting effort, this might be ok.)
After a bunch of troubleshooting and debugging on my end with manually calling the API, it looks like the reason for the 400 bad request is due to the image format. Ollama does not currently support webp, and only supports jpg and png for now: https://github.com/ollama/ollama/issues/2457.
Not sure the feasibility, but would it be possible to either do png/jpg instead of webp, or maybe see that the "OPENAI_BASE_URL" env variable is set, so convert from webp to png/jpg before passing to Ollama?
michael-genson
commented
2 weeks ago Huh, weird. Okay yeah that's an easy fix, I only chose webp because we already convert images to webp in Mealie
death2all110
commented
2 weeks ago Thank you! I'll keep an eye on it and see if I can test once its merged.
death2all110
commented
2 weeks ago @boc-the-git @michael-genson would it be possible to get this put into a stable build? like a v1.12.1? Not sure if there are any plans for a another stable build before 2.0 with the households feature (which will break some things for me so I'm currently not on the nightly)
lowlyocean
commented
2 weeks ago Although the initial error has gone away, I'm still seeing 500 Internal Server Error. Has anyone got this working? If so can you share what version Ollama and what model you're using?
death2all110
commented
2 weeks ago I haven't had a chance to test this, but when I was testing with cURL, debug logs from Ollama mentioned the context was too long so was truncated, I then found bakllava had a larger context size than standard llava, so might try that @lowlyocean
lowlyocean
commented
1 week ago Tried bakllava 7b q4_K_S , , I just keep getting response "no slots available after 10 retries" from Ollama on most attempts, but I saw (only one time) that it returned some JSON reply, having nonsense values. I think the prompt is too large to work on a local model.
If I remove the System role message from the request Mealie generates, it replies more often - but every response is nonsense (and not JSON formatted).
death2all110
commented
1 week ago Yes, I tried bakllava as well and it didn't work. Using standard llava, but creating a model file with an increased context size 4096 instead of standard 2048, worked. Unfortunately, it still wasn't 100% and would take liberties with what would be generated. (Example peanut butter fudge recipe turned into peanut butter cookies). At this point I'm waiting for ollama to get support for Phi 3.5-Vision which seems impressive from what I have seen. And I hear MiniCPM-V is decent as well, but ollama also has no support for that. Not sure of any other vision models that aren't llava based.
First Check
What is the issue you are experiencing?
When Mealie is configured with Ollama instead of OpenAI, scraping from image does not work.
Steps to Reproduce
Please provide relevant logs
Mealie Version
Image: hkotel/mealie:latest Version: v1.12.0 Build: 0d06494bbfc62fc8a1771a85fd308190891d6167
Deployment
Unraid
Additional Deployment Details
Image used for testing: