search

mealie-recipes / mealie

Mealie is a self hosted recipe manager and meal planner with a RestAPI backend and a reactive frontend application built in Vue for a pleasant user experience for the whole family. Easily add recipes into your database by providing the url and mealie will automatically import the relevant data or add a family recipe with the UI editor
https://docs.mealie.io
GNU Affero General Public License v3.0
7.29k stars 727 forks source link

[BUG] - Create Recipe from an Image fails Pydantic validation #4286

Closed rhardih closed 1 month ago

rhardih commented 1 month ago

First Check

What is the issue you are experiencing?

Creating a recipe from an image fails.

Steps to Reproduce

  1. Got to https:///g/home/r/create/image
  2. Upload image.
  3. Click "+ Create"

Test image:

flaeskestegsandwich

Please provide relevant logs

Traceback (most recent call last): File "/app/mealie/services/recipe/recipe_service.py", line 468, in build_recipe_from_images openai_recipe = OpenAIRecipe.parse_openai_response(response) File "/app/mealie/schema/openai/_base.py", line 24, in parse_openai_response return cls.model_validate_json(response or "") File "/opt/pysetup/.venv/lib/python3.10/site-packages/pydantic/main.py", line 597, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) pydantic_core._pydantic_core.ValidationError: 2 validation errors for OpenAIRecipe name Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.8/v/missing description Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.8/v/missing

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi result = await app( # type: ignore[func-returns-value] File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call return await self.app(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/applications.py", line 123, in call await self.middleware_stack(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in call raise exc File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in call await self.app(scope, receive, _send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 24, in call await responder(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 44, in call await self.app(scope, receive, self.send_with_gzip) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 756, in call await self.middleware_stack(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle await self.app(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app ERROR 2024-09-30T09:53:38 - Exception in ASGI application Traceback (most recent call last): File "/app/mealie/services/recipe/recipe_service.py", line 468, in build_recipe_from_images openai_recipe = OpenAIRecipe.parse_openai_response(response) File "/app/mealie/schema/openai/_base.py", line 24, in parse_openai_response return cls.model_validate_json(response or "") File "/opt/pysetup/.venv/lib/python3.10/site-packages/pydantic/main.py", line 597, in model_validate_json return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context) pydantic_core._pydantic_core.ValidationError: 2 validation errors for OpenAIRecipe response = await func(request) File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler response = await original_route_handler(request) File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 289, in create_recipe_from_image recipe = await self.service.create_from_images(images, translate_language) File "/app/mealie/services/recipe/recipe_service.py", line 270, in create_from_images recipe_data = await openai_recipe_service.build_recipe_from_images( File "/app/mealie/services/recipe/recipe_service.py", line 471, in build_recipe_from_images raise ValueError("Unable to parse recipe from image") from e ValueError: Unable to parse recipe from image name Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.8/v/missing description Field required [type=missing, input_value={}, input_type=dict] For further information visit https://errors.pydantic.dev/2.8/v/missing

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 401, in run_asgi result = await app( # type: ignore[func-returns-value] File "/opt/pysetup/.venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call return await self.app(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/applications.py", line 123, in call await self.middleware_stack(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in call raise exc File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in call await self.app(scope, receive, _send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 24, in call await responder(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 44, in call await self.app(scope, receive, self.send_with_gzip) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 756, in call await self.middleware_stack(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle await self.app(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/opt/pysetup/.venv/lib/python3.10/site-packages/starlette/routing.py", line 72, in app response = await func(request) File "/app/mealie/routes/_base/routers.py", line 35, in custom_route_handler response = await original_route_handler(request) File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( File "/opt/pysetup/.venv/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "/app/mealie/routes/recipe/recipe_crud_routes.py", line 289, in create_recipe_from_image recipe = await self.service.create_from_images(images, translate_language) File "/app/mealie/services/recipe/recipe_service.py", line 270, in create_from_images recipe_data = await openai_recipe_service.build_recipe_from_images( File "/app/mealie/services/recipe/recipe_service.py", line 471, in build_recipe_from_images raise ValueError("Unable to parse recipe from image") from e ValueError: Unable to parse recipe from image

Mealie Version

ghcr.io/mealie-recipes/mealie:v1.12.0

Deployment

Other (please specify below)

Additional Deployment Details

Deployed on K3s.

I've tested both gpt-4o and gpt-4o-mini - I'm unsure if it even matters, as I imagine the validation happens even before.

Kuchenpirat commented 1 month ago

can you provide the image for us to test?

rhardih commented 1 month ago

Of course. I've attached it to the original reproduction steps.

Kuchenpirat commented 1 month ago

Oh ok, now i see what is happening. This feature is not meant to create a recipe from an image of a dish but from an image of a recipe with an ingredient list and instructions.

But the burger looks pretty good, would not mind a recipe for that 😋

rhardih commented 1 month ago

I didn't realise that. Accidental feature request maybe. 😆

It's definitely possible. I just tested with Claude for instance, with the same picture and the prompt: "Can you make a recipe for this picture?". This is what it came up with:

Based on the image, I can provide a recipe for what appears to be a gourmet sandwich or burger. Here's a recipe inspired by the components visible in the picture:

Gourmet Sesame Bun Sandwich

Ingredients:
- 1 sesame seed bun
- 2-3 slices of roasted or smoked turkey
- 2-3 thin slices of apple
- 2-3 slices of red onion
- A handful of arugula leaves
- 1-2 pickles, sliced
- 2-3 crispy fried onion strings
- Mayonnaise or aioli (optional)
- Salt and pepper to taste

Instructions:
1. Lightly toast the sesame seed bun.
2. If using, spread a thin layer of mayonnaise or aioli on both halves of the bun.
3. On the bottom half of the bun, layer the ingredients in this order:
   - A few arugula leaves
   - Turkey slices
   - Apple slices
   - Red onion slices
   - Pickle slices
4. Sprinkle salt and pepper to taste.
5. Top with crispy fried onion strings.
6. Add a few more arugula leaves on top.
7. Place the top half of the bun to complete the sandwich.
8. Serve immediately and enjoy!

This recipe combines the savory flavors of turkey with the crunch and sweetness of apple, the tang of pickles, and the peppery bite of arugula. The crispy onion strings add texture, while the sesame seed bun provides a nutty flavor to complement the filling.

Would you like me to elaborate on any part of the recipe or suggest any variations?
Kuchenpirat commented 1 month ago

I don't know how reliable this would be but you are more than welcome to open a Feature Request for it (in the Discussions)