[BUG] - Create from image returns "Something Went Wrong!"/ "Unable to parse recipe from image"

[realthore]

First Check

• [X] This is not a feature request.
• [X] I added a very descriptive title to this issue (title field is above this).
• [X] I used the GitHub search to find a similar issue and didn't find it.
• [X] I searched the Mealie documentation, with the integrated search.
• [X] I already read the docs and didn't find an answer.
• [ ] This issue can be replicated on the demo site (https://demo.mealie.io/).

What is the issue you are experiencing?

I'm trying to create an image from a recipe. Tried multiple different images, either handwritten and photos of printed recipes. All return with "Something Went Wrong!"

Steps to Reproduce

Upload image to mealie, error occurs after a couple of seconds it's loading. OpenAI API usage activity doesn't change

Please provide relevant logs

 File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler
    response = await original_route_handler(request)
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 289, in create_recipe_from_image
    recipe = await self.service.create_from_images(images, translate_language)
  File "/app/mealie/services/recipe/recipe_service.py", line 270, in create_from_images
    recipe_data = await openai_recipe_service.build_recipe_from_images(
  File "/app/mealie/services/recipe/recipe_service.py", line 471, in build_recipe_from_images
    raise ValueError("Unable to parse recipe from image") from e
ValueError: Unable to parse recipe from image

Docker compose:

      - OPENAI_BASE_URL=https://api.openai.com/
      - OPENAI_API_KEY= my key
      - OPENAI_MODEL=gpt-4o
      - OPENAI_ENABLE_IMAGE_SERVICE=true
      - OPENAI_WORKERS=2
      - OPENAI_SEND_DATABASE_DATA=true
      - OPENAI_REQUEST_TIMEOUT=60

Mealie Version

https://github.com/mealie-recipes/mealie/commit/9c04950948428f333dea5e21575b66eef7ec0e05

Deployment

Docker (Synology)

Additional Deployment Details

Using Portainer

[michael-genson]

Are you able to provide the image you used? Also, are you on the free tier or paid tier in OpenAI?

[realthore]

I'm on the free tier, but haven't used chatgpt in the last 24h so I shouldn't have exceeded the GPT4o limit

[michael-genson]

I'm not sure gpt-4o works on the free tier: https://platform.openai.com/docs/guides/rate-limits/usage-tiers?context=tier-free

I was able to create the recipe on my instance using Tier 1:

(it put all the instructions in one step annoyingly)

[realthore]

Okay got it. I'll upgrade my API tier in the next couple of days, thanks for your assistance

[megosugit]

Hi, same issue here, but I even tried with the simple examples in Parser, and for instance it fails to evaluate: 2 tbsp minced cilantro, leaves and stems with the same error: "Something Went Wrong!"

Never used openai before, I just created my account and an API key. Mealie claims: OpenAI Ready Required OpenAI variables are all set.

[Kuchenpirat]

If you set the LOG_LEVEL to debug there will be more information to why the job has failed

https://docs.mealie.io/documentation/getting-started/installation/backend-config/#general

[megosugit]

I found this: openai.NotFoundError: Error code: 404 - {'error': {'message': 'The model gpt-4o does not exist or you do not have access to it.', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}

I then switched to gpt-4o-mini and now I get this:

openai.RateLimitError: Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}}

I'll try to sort it out with openai directly, it is not related to Mealie :)

thanks

[Kuchenpirat]

Yeah, gpt-4o might not be included in the free tier.

For the other one. You will have to buy API tokens from OpenAI to resolve this. From my experience it is about 0.5 cents to create a recipe and about 0.1 cents for ingredient parsing so it certainly is pretty cost effective. You milage might vary but it should not bancrupt you and you will propably get pretty far with the min amount of 5$.

[megosugit]

shouldn't the free tier be enough ? it should offer $100 / month

I'll check with them first, because when I created my account, it said I won't get free credit because my number has been used before. Which is weird as I never used Openai before...

What is also surprising, is it claims this API key has never been used, as if it didn't detect Mealie attempt.

[michael-genson]

The free tier limits are too low to work with any real-world application. You either need to go up to Tier 1 or use an open source/self hosted model.

If you put $5 into OpenAI they'll bump you up to Tier 1, which gives you gpt-4 models and substantially higher limits. I recognize that this isn't an option for everyone, in which case I would recommend trying out ollama. But you'll need pretty beefy hardware to get good results out of it.

[megosugit]

Maybe, I never tried so far, how long will it last with "normal" usage ? and does it expire after some months ?