mealie-recipes / mealie

Mealie is a self hosted recipe manager and meal planner with a RestAPI backend and a reactive frontend application built in Vue for a pleasant user experience for the whole family. Easily add recipes into your database by providing the url and mealie will automatically import the relevant data or add a family recipe with the UI editor
https://docs.mealie.io
GNU Affero General Public License v3.0
7.17k stars 719 forks source link

Scrapped recipe has no photo and can't be updated to include one #153

Closed W1ndst0rm closed 3 years ago

W1ndst0rm commented 3 years ago

Describe the bug I tried importing a by URL and everything except the image came through fine. I tried to set the image manually but that didn't work either.

Steps To Reproduce

  1. From the homepage open the import from URL dialog
  2. Enter https://altonbrown.com/recipes/homemade-soft-pretzels/
  3. Click submit

The resulting recipe doesn't have an image. If you try to assign one by editing the recipe it fails.

Expected Behavior The web scrapper should pull in the image correctly, but if it doesn't I should be able to edit it manually and fix it.

Device Information (please complete the following information):

Additional context When I checked the recipe in the API the image key is set to "no image"

Log entry from trying to add the recipe


10-Feb-21 08:32:44 INFO: Recipe Scraped From Web: {'@context': 'http://schema.org/', '@type': 'Recipe', 'name': 'Homemade Soft Pretzels', 'author': {'@type': 'Person', 'name': 'Level Agency'}, 'description': 'Buttery soft pretzels are achievable at home with a two-stage cooking method and plenty of salt. Although there is such a thing as salt-less pretzels called “baldies,” I would suggest that they’re really not pretzels at all, but rather cruel jokes perpetrated by bitter bakers. This recipe first appeared in Season 11 of Good Eats.', 'datePublished': datetime.datetime(2020, 8, 21, 16, 54, 48, tzinfo=datetime.timezone.utc), 'image': ['https://altonbrown.com/wp-content/uploads/2020/08/Homemade-Soft-Pretzels_resized.jpg', 'https://altonbrown.com/wp-content/uploads/2020/08/Homemade-Soft-Pretzels_resized-500x500.jpg', 'https://altonbrown.com/wp-content/uploads/2020/08/Homemade-Soft-Pretzels_resized-500x375.jpg', 'https://altonbrown.com/wp-content/uploads/2020/08/Homemade-Soft-Pretzels_resized-480x270.jpg'], 'recipeYield': ['8', '8 pretzels'], 'prepTime': datetime.timedelta(seconds=1800), 'totalTime': datetime.timedelta(seconds=6300), 'recipeIngredient': ['1 1/2 cups warm water, 110-115ºF, plus 10 cups water for boiling', '1 tablespoon sugar', '2 teaspoons kosher salt', '1 package active dry yeast', '4 1/2 cups all-purpose flour', '4 tablespoons unsalted butter (melted)', '2/3 cup baking soda', '1 large egg yolk, beaten with 1 tablespoon water', 'Vegetable oil, for the bowl and pan', 'Pretzel salt'], 'recipeInstructions': [{'@type': 'HowToStep', 'text': 'Combine the 1 1/2 cups warm water, the sugar, and kosher salt in the bowl of a stand mixer and sprinkle the yeastSoft pretzels are always made with yeast dough and are very similar to bagels. on top. Set aside until the mixture foams, about 5 minutes.', 'name': 'Combine the 1 1/2 cups warm water, the sugar, and kosher salt in the bowl of a stand mixer and sprinkle the yeastSoft pretzels are always made with yeast dough and are very similar to bagels. on top. Set aside until the mixture foams, about 5 minutes.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-0'}, {'@type': 'HowToStep', 'text': 'Add the flour and butter and, using the dough hook attachment, mix on low speed until well combined. Change to medium speed and knead until the dough is smooth and pulls away from the side of the bowl, 4 to 5 minutes.', 'name': 'Add the flour and butter and, using the dough hook attachment, mix on low speed until well combined. Change to medium speed and knead until the dough is smooth and pulls away from the side of the bowl, 4 to 5 minutes.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-1'}, {'@type': 'HowToStep', 'text': 'Remove the dough from the bowl, clean the bowl, then oil it well. Return the dough to the bowl, cover with plastic wrap, and set aside in a warm place until the dough has doubled in size, 50 to 55 minutes.', 'name': 'Remove the dough from the bowl, clean the bowl, then oil it well. Return the dough to the bowl, cover with plastic wrap, and set aside in a warm place until the dough has doubled in size, 50 to 55 minutes.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-2'}, {'@type': 'HowToStep', 'text': 'Heat oven to 450ºF. Line 2 half-sheet pans with parchment paper and lightly brush with oil, about 1 tablespoon per pan. Set aside.', 'name': 'Heat oven to 450ºF. Line 2 half-sheet pans with parchment paper and lightly brush with oil, about 1 tablespoon per pan. Set aside.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-3'}, {'@type': 'HowToStep', 'text': 'Bring the remaining 10 cups water and the baking soda to a rolling boil in a Dutch oven.', 'name': 'Bring the remaining 10 cups water and the baking soda to a rolling boil in a Dutch oven.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-4'}, {'@type': 'HowToStep', 'text': 'Meanwhile, turn the dough out onto a lightly oiled work surface and divide into 8 equal pieces. Roll out each piece of dough into a 24-inch rope. Make a U-shape with the rope, and, holding the ends of the rope, cross them over each other and press onto the bottom of the U in order to form the shape of a pretzel. Place on a half-sheet pan. Repeat with the remaining pieces of dough.', 'name': 'Meanwhile, turn the dough out onto a lightly oiled work surface and divide into 8 equal pieces. Roll out each piece of dough into a 24-inch rope. Make a U-shape with the rope, and, holding the ends of the rope, cross them over each other and press onto the bottom of the U in order to form the shape of a pretzel. Place on a half-sheet pan. Repeat with the remaining pieces of dough.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-5'}, {'@type': 'HowToStep', 'text': 'One by one, place the pretzels in the boiling water for 30 seconds. Remove them from the water using a large flat spatula. Return them to the sheet pans, brush the top of each pretzel with the beaten egg yolk and water mixture, and sprinkle with pretzel salt.If you don’t have pretzel salt, coarse sea salt will do.', 'name': 'One by one, place the pretzels in the boiling water for 30 seconds. Remove them from the water using a large flat spatula. Return them to the sheet pans, brush the top of each pretzel with the beaten egg yolk and water mixture, and sprinkle with pretzel salt.If you don’t have pretzel salt, coarse sea salt will do.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-6'}, {'@type': 'HowToStep', 'text': 'Bake until dark golden brown in color, 12 to 14 minutes. Transfer to a wire rack for at least 5 minutes before serving.', 'name': 'Bake until dark golden brown in color, 12 to 14 minutes. Transfer to a wire rack for at least 5 minutes before serving.', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#wprm-recipe-5484-step-0-7'}], 'aggregateRating': {'@type': 'AggregateRating', 'ratingValue': '4.53', 'ratingCount': '61'}, 'recipeCategory': ['Breads'], 'keywords': 'Baking, Game Day, Hacks, Homemade Soft Pretzels, Snacks', '@id': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#recipe', 'isPartOf': {'@id': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#webpage'}, 'mainEntityOfPage': 'https://altonbrown.com/recipes/homemade-soft-pretzels/#webpage', 'url': 'https://altonbrown.com/recipes/homemade-soft-pretzels/'}
INFO:     172.17.0.1:45560 - "POST /api/recipes/create-url HTTP/1.1" 201 Created
INFO:     172.17.0.1:45560 - "GET /api/recipes?keys=name&keys=slug&keys=image&keys=description&keys=dateAdded&keys=rating&num=100 HTTP/1.1" 200 OK
INFO:     172.17.0.1:45560 - "GET /api/recipes/homemade-soft-pretzels HTTP/1.1" 200 OK
INFO:     172.17.0.1:45560 - "GET /api/recipes/no%20image/image?rnd=1 HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 396, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 111, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 566, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 227, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 41, in app
    response = await func(request)
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 201, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 150, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.9/site-packages/starlette/concurrency.py", line 34, in run_in_threadpool
    return await loop.run_in_executor(None, func, *args)
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "./routes/recipe/recipe_crud_routes.py", line 73, in get_recipe_img
    return FileResponse(recipe_image)
  File "/usr/local/lib/python3.9/site-packages/starlette/responses.py", line 257, in __init__
    media_type = guess_type(filename or path)[0] or "text/plain"
  File "/usr/local/lib/python3.9/mimetypes.py", line 292, in guess_type
    return _db.guess_type(url, strict)
  File "/usr/local/lib/python3.9/mimetypes.py", line 116, in guess_type
    url = os.fspath(url)
TypeError: expected str, bytes or os.PathLike object, not NoneType```
hay-kot commented 3 years ago

I was able to find the bug. After some refactoring it seems I missed passing an additional parameter when calling to update the image. I'll need to do some additional checks but I should be able to get a patch out in the next day or two, Thanks!

W1ndst0rm commented 3 years ago

Tested after updating to #159 and I was able to add images again. Great work

hay-kot commented 3 years ago

Wohoo, Thanks for closing the issue! 🎉