Unable to import recipe from JSON source

agc93 commented 1 year ago

Tandoor Version

1.4.8

Setup

Docker / Docker-Compose

Reverse Proxy

Traefik

Other

No response

Bug description

Attempting to import a recipe with the following JSON results in an error, although the recipe seems to be well-formed from a check agains the JSON Recipe schema on schema.org. The import is being attempted from the Source tab of the Import Recipe page.

Recipe JSON

```json { "name": "Easy Smoked Salmon Sushi Bowl", "description": "Looking for dinner on the table quick smart? These fast recipes are ready in 25 minutes or less and all come with yumness guaranteed.", "totalTime": "PT0H25M0S", "recipeCategory": "fish", "keywords": [ ], "recipeIngredient": [ "1 spring onion", "1 carrot", "150g jasmine rice", "1 avocado", "100g smoked salmon slices", "40g Japanese dressing", "1 garlic clove", "1 tsp white vinegar", "1 tsp sugar", "2 eggs" ], "recipeInstructions": [ { "text": "Check your avocado is ripe before starting this recipe (see Kitchen hack). Thinly slice 1 garlic clove. Thinly slice the spring onion. Peel the carrot into ribbons with a vegetable peeler.\n", "name": "Prep ingredients ", "@type": "HowToStep" }, { "text": "Rinse the rice until the water runs clear. Put in a small saucepan with the garlic, half the spring onion and 250ml (1 cup) water, cover and bring to a simmer over medium heat. Reduce the heat to low and cook for 12 mins or until tender and the water is absorbed. Turn off the heat and stand, covered, for at least 5 mins. Season with salt and pepper. Fluff the rice with a fork.\n", "name": "Cook rice ", "@type": "HowToStep" }, { "text": "While the rice is cooking, bring a small saucepan of water to the boil for the eggs. Combine 1 tsp white vinegar and 1 tsp sugar in a large bowl. Add the carrot and toss to combine. Set aside to pickle, tossing occasionally, until needed.\n", "name": "Pickle carrot", "@type": "HowToStep" }, { "text": "Cook 2 eggs in the pan of boiling water for 6 mins for soft-boiled or 8 mins for hard-boiled. Drain, then cool in cold water. Peel and halve the eggs.\n", "name": "Cook eggs", "@type": "HowToStep" }, { "text": "Thinly slice the avocado. Drain the carrot, discarding the pickling liquid. Separate the smoked salmon slices, then coarsely tear into pieces. Divide the rice among bowls and top with the avocado, pickled carrot, salmon and egg. Drizzle with the Japanese dressing and scatter over the remaining spring onion. Enjoy!\n", "name": "Serve up ", "@type": "HowToStep" }, { "text": "To speed up the ripening process, put the avocado in a paper bag and loosely close, then leave on the kitchen bench for a day or two. Add a banana to the bag to make the avocado ripen faster.\n", "name": "Kitchen hack", "@type": "HowToStep" } ], "nutrition": { "calories": "625kcal", "carbohydrateContent": "69.2g", "proteinContent": "23.9g", "fatContent": "26.1g", "@type": "NutritionInformation" }, "recipeYield": 2, "dateCreated": "2023-03-27T00:32:10.324Z" } ```

The logs indicate a key error when updating the source_url property of the JSON, which confuses me since that's not a known property of the Recipe schema. Does the recipe JSON need to be embedded into a HTML page to be imported?

Relevant logs

tandoor-nginx_1  | 172.26.0.5 - - [19/Apr/2023:00:09:56 +0000] "POST /api/recipe-from-source/ HTTP/1.1" 500 145 "https://kitchen.agc93.au/data/import/url" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36 Edg/112.0.1722.46" "172.69.62.9"
tandoor_1        |   File "/opt/recipes/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 103, in view
tandoor_1        |     return self.dispatch(request, *args, **kwargs)
tandoor_1        |   File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
tandoor_1        |     response = self.handle_exception(exc)
tandoor_1        |   File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
tandoor_1        |     self.raise_uncaught_exception(exc)
tandoor_1        |   File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
tandoor_1        |     raise exc
tandoor_1        |   File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
tandoor_1        |     response = handler(request, *args, **kwargs)
tandoor_1        |   File "/opt/recipes/venv/lib/python3.10/site-packages/rest_framework/decorators.py", line 50, in handler
tandoor_1        |     return func(*args, **kwargs)
tandoor_1        |   File "/opt/recipes/cookbook/views/api.py", line 1224, in recipe_from_source
tandoor_1        |     'recipe_json': helper.get_from_scraper(scrape, request),
tandoor_1        |   File "/opt/recipes/cookbook/helper/recipe_url_import.py", line 194, in get_from_scraper
tandoor_1        |     if recipe_json['source_url']:
tandoor_1        | KeyError: 'source_url'

vabene1111 commented 1 year ago

its something that usually comes with a recipe when its imported, need to implement a check if it comes from json so its not requested.

thanks for the good report

agc93 commented 1 year ago

I can see in the release notes for v1.4.12 a mention of a fix for an issue that sounds similar to this, does that mean this is fixed?

vabene1111 commented 1 year ago

yes, i just did not see that this issue exists, thansk for pointing it out.

agc93 commented 1 year ago

@vabene1111 I just updated to v1.5.0 and using the same JSON recipe as the initial report now fails with a different error message.

There's no traceback, and the POST to /api/recipe now returns a 400 with the response as below:

{"name":["This field may not be blank."]}

Is this just a new message for the same issue or a new problem?

EDIT: I've tested a bit more and the same JSON recipe will fail to import when imported using the Import Recipe -> Source option, but will work when added to a ZIP and imported using the "Nextcloud Cookbooks" import type.

vabene1111 commented 1 year ago

intersting, i dont get an error but it does not load any data from your json, need to take a look at that, might have changed something that broke this again.

vabene1111 commented 1 year ago

i am not sure exactly but i can import this json without any problems

need to look into why yours above is not working but in the end we likely need to fix this in because we use their text scraper and it does not return anything on your input https://github.com/hhursev/recipe-scrapers

[
   {
      "name":"Schokoladenpudding Rezept",
      "description":" Schokoladenpudding: Zartbitterschokolade, Milch, Speisestärke, Kakaopulver, Zucker, Schlagsahne",
      "image":[
         {
            "url":"https://image.essen-und-trinken.de/11919168/t/77/v8/w1440/r1/-/schokoladenpudding-373f84f3b0e51409f1fc50386cdaff0a-fjt2013100301-jpg--7243-.jpg",
            "width":"1440",
            "height":"1440",
            "@type":"ImageObject"
         },
         {
            "url":"https://image.essen-und-trinken.de/11919168/t/Fc/v8/w1440/r1.7778/-/schokoladenpudding-373f84f3b0e51409f1fc50386cdaff0a-fjt2013100301-jpg--7243-.jpg",
            "width":"1440",
            "height":"810",
            "@type":"ImageObject"
         },
         {
            "url":"https://image.essen-und-trinken.de/11919168/t/KI/v8/w1440/r1.5/-/schokoladenpudding-373f84f3b0e51409f1fc50386cdaff0a-fjt2013100301-jpg--7243-.jpg",
            "width":"1440",
            "height":"960",
            "@type":"ImageObject"
         }
      ],
      "mainEntityOfPage":{
         "@id":"https://www.essen-und-trinken.de/rezepte/58294-rzpt-schokoladenpudding",
         "@type":"WebPage"
      },
      "datePublished":"2013-10-15T13:19:00+02:00",
      "dateModified":"2020-09-26T14:08:12+02:00",
      "author":{
         "name":"Gelinggarantie",
         "@type":"Person"
      },
      "publisher":{
         "name":"ESSEN-UND-TRINKEN.de",
         "logo":{
            "url":"https://image.essen-und-trinken.de/12189640/uncropped-0-0/28f90a0670f68430ae63d64d0321b5af/gi/eut-logo60px.png",
            "width":"130",
            "height":"60",
            "@type":"ImageObject"
         },
         "@type":"Organization"
      },
      "aggregateRating":{
         "ratingCount":169,
         "bestRating":5,
         "worstRating":1,
         "ratingValue":3.8,
         "@type":"AggregateRating"
      },
      "totalTime":"PT10M",
      "recipeYield":4,
      "recipeIngredient":[
         "100 g Zartbitterschokolade",
         "500 ml Milch",
         "35 g Speisestärke",
         "2 El Kakaopulver",
         "50 g Zucker",
         "100 ml Schlagsahne"
      ],
      "recipeInstructions":[
         "Schokolade grob hacken. 350 ml Milch in einem Topf erwärmen. Die Hälfte der Schokolade zugeben und unter Rühren schmelzen.",
         "Die restliche Milch mit Stärke, Kakao und Zucker mit dem Schneebesen klümpchenfrei verrühren. Schokoladenmilch unter Rühren zum Kochen bringen. Kakao-Stärke-Mischung einrühren, aufkochen und 1 Min. kochen. Heißen Pudding in 4 Tassen füllen (Inhalt ca. 150 ml), mit Klarsichtfolie abdecken und kalt stellen.",
         "Sahne steif schlagen. Schokoladenpudding mit Sahne und der restlichen gehackten Schokolade bestreut servieren."
      ],
      "nutrition":{
         "calories":375,
         "proteinContent":7,
         "fatContent":20,
         "carbohydrateContent":38,
         "@type":"NutritionInformation"
      },
      "@context":"http://schema.org",
      "@type":"Recipe"
   },
   {
      "@context":"http://schema.org",
      "@type":"BreadcrumbList",
      "itemListElement":[
         {
            "position":1,
            "item":{
               "name":"essenundtrinken",
               "@id":"https://www.essen-und-trinken.de/",
               "@type":"WebPage"
            },
            "@type":"ListItem"
         },
         {
            "position":2,
            "item":{
               "name":"Rezepte",
               "@id":"https://www.essen-und-trinken.de/rezepte/",
               "@type":"WebPage"
            },
            "@type":"ListItem"
         },
         {
            "position":3,
            "item":{
               "name":"Schokoladenpudding Rezept",
               "@id":"https://www.essen-und-trinken.de/rezepte/58294-rzpt-schokoladenpudding",
               "@type":"WebPage"
            },
            "@type":"ListItem"
         }
      ]
   }
]

vabene1111 commented 1 year ago

ok its your json is missing the

      "@context":"http://schema.org",
      "@type":"Recipe"

line, I will an additonal check to add this exact type hint if nothign is given since this should be everything people post to the endpoint anyway.

agc93 commented 1 year ago

Oh that's a good spot, I didn't even think of the top-level schema specification, but adding that does import perfectly.

vabene1111 commented 1 year ago

auto adding these in the next release

TandoorRecipes / recipes