hhursev / recipe-scrapers

Python package for scraping recipes data
MIT License
1.62k stars 508 forks source link

Ingredient Grouping Missing, www.joshuaweissman.com #985

Open Kvan7 opened 6 months ago

Kvan7 commented 6 months ago

Pre-filing checks

The URL of the recipe(s) that are not being scraped correctly

...

The results you expect to see Recipe groups filled as similar to section headings. This is from the way these are being parsed currently, the css selector for headers is also grabbing the headers in instructions and when they are the same, it sets that dict to an empty list. See group_ingredients from _grouping_utils.py:

       if el in el.parent.select(group_heading):
            # This is the heading
            current_heading = normalize_string(el.get_text())
            groupings[current_heading] = []

This is not an issue with the function, this website just is hard to parse and the best way to grab the group headings also grabs the headings from the instructions/methods. When there are the same headings in both, the instruction part overwrites the list and sets it to empty.

...

The results (including any Python error messages) that you are seeing

Abbreviated log:

    "ingredient_groups": [
        {
            "ingredients": [
                "1 cup (240ml) buttermilk",
                "1 egg",
                "1 egg yolk",
                "2 cups (300g) all-purpose flour",
                "\u00bd cup (120ml) milk",
                "\u00bd cup (95g) fresh blueberries",
                "1/4 cup (62g) granulated sugar",
                "1 teaspoon (4g) fine sea salt",
                "1/4 teaspoon (1g) baking soda",
                "1 tablespoon (12g) baking powder",
                "2 tablespoon (21g) vegetable oil",
                "1 tsp (4g) vanilla extract, optional",
                "Whipped cream to serve"
            ],
            "purpose": "Blueberry Buttermilk Pancake"
        },
        {
            "ingredients": [
                "1 cup (225g) brown sugar",
                "1 Tbsp (14g) muscovado sugar",
                "\u00bd cup (120ml) water",
                "\u00bd cup (120ml) maple syrup",
                "2.5 Tbsp (17g) cinnamon",
                "1/3 cup (84g) butter",
                "1 teaspoon (7g) fine sea salt"
            ],
            "purpose": "Cinnamon Spread"
        },
        {
            "ingredients": [
                "4 oz (115g) cream cheese",
                "3/4 cups (90g) powdered sugar",
                "1 vanilla bean",
                "2 tablespoons (35g) whole milk"
            ],
            "purpose": "Vanilla Cream Cheese Glaze"
        }
    ],
    "ingredients": [
        "1 cup (240ml) buttermilk",
        "1 egg",
        "1 egg yolk",
        "2 cups (300g) all-purpose flour",
        "\u00bd cup (120ml) whole milk",
        "1/4 cup (62g) granulated sugar",
        "1 teaspoon (4g) fine sea salt",
        "1/4 teaspoon (1g) baking soda",
        "1 tablespoon (12g) baking powder",
        "2 tablespoon (21g) vegetable oil",
        "1 tsp (4g) vanilla extract, optional",
        "Butter to finish",
        "Maple syrup to finish",
        "1 cup (240ml) buttermilk",
        "1 egg",
        "1 egg yolk",
        "2 cups (300g) all-purpose flour",
        "\u00bd cup (120ml) milk",
        "\u00bd cup (95g) fresh blueberries",
        "1/4 cup (62g) granulated sugar",
        "1 teaspoon (4g) fine sea salt",
        "1/4 teaspoon (1g) baking soda",
        "1 tablespoon (12g) baking powder",
        "2 tablespoon (21g) vegetable oil",
        "1 tsp (4g) vanilla extract, optional",
        "Whipped cream to serve",
        "2 \u00bd cups (285g) fresh blueberries",
        "1 cup (200g) granulated white sugar",
        "\u00bd cup (120ml) water",
        "\u00bc cup (60ml) maple syrup",
        "1 tablespoon (17g) light corn syrup",
        "3 tablespoons (55g) tablespoon lemon juice",
        "Whipped Cream:",
        "1 cup (240ml) heavy whipping cream",
        "2 tablespoons (14g) powdered sugar",
        "1 tablespoon (15g) creme fraiche, optional",
        "1 cup (240ml) buttermilk",
        "1 egg",
        "1 egg yolk",
        "2 cups (300g) all-purpose flour",
        "\u00bd cup (120ml) whole milk",
        "1/4 cup (62g) granulated sugar",
        "1 teaspoon (4g) fine sea salt",
        "1/4 teaspoon (1g) baking soda",
        "1 tablespoon (12g) baking powder",
        "2 tablespoon (21g) vegetable oil",
        "1 tsp (4g) vanilla extract, optional",
        "Whipped cream to serve",
        "1 cup (225g) brown sugar",
        "1 Tbsp (14g) muscovado sugar",
        "\u00bd cup (120ml) water",
        "\u00bd cup (120ml) maple syrup",
        "2.5 Tbsp (17g) cinnamon",
        "1/3 cup (84g) butter",
        "1 teaspoon (7g) fine sea salt",
        "4 oz (115g) cream cheese",
        "3/4 cups (90g) powdered sugar",
        "1 vanilla bean",
        "2 tablespoons (35g) whole milk"
    ],

Full

{"author": "Lawrence Weibman", "canonical_url": "https://www.joshuaweissman.com/post/making-ihop-pancakes-at-home", "category": null, "description": "It\u2019s MY House of Pancakes now.", "host": "joshuaweissman.com", "image": "https://static.wixstatic.com/media/10e36c_804539f862bb47bba963eab6e7937805~mv2.jpg/v1/fill/w_1000,h_563,al_c,q_85,usm_0.66_1.00_0.01/10e36c_804539f862bb47bba963eab6e7937805~mv2.jpg ", "ingredient_groups": [{"ingredients": ["1 cup (240ml) buttermilk", "1 egg", "1 egg yolk", "2 cups (300g) all-purpose flour", "\u00bd cup (120ml) milk", "\u00bd cup (95g) fresh blueberries", "1/4 cup (62g) granulated sugar", "1 teaspoon (4g) fine sea salt", "1/4 teaspoon (1g) baking soda", "1 tablespoon (12g) baking powder", "2 tablespoon (21g) vegetable oil", "1 tsp (4g) vanilla extract, optional", "Whipped cream to serve"], "purpose": "Blueberry Buttermilk Pancake"}, {"ingredients": ["1 cup (225g) brown sugar", "1 Tbsp (14g) muscovado sugar", "\u00bd cup (120ml) water", "\u00bd cup (120ml) maple syrup", "2.5 Tbsp (17g) cinnamon", "1/3 cup (84g) butter", "1 teaspoon (7g) fine sea salt"], "purpose": "Cinnamon Spread"}, {"ingredients": ["4 oz (115g) cream cheese", "3/4 cups (90g) powdered sugar", "1 vanilla bean", "2 tablespoons (35g) whole milk"], "purpose": "Vanilla Cream Cheese Glaze"}], "ingredients": ["1 cup (240ml) buttermilk", "1 egg", "1 egg yolk", "2 cups (300g) all-purpose flour", "\u00bd cup (120ml) whole milk", "1/4 cup (62g) granulated sugar", "1 teaspoon (4g) fine sea salt", "1/4 teaspoon (1g) baking soda", "1 tablespoon (12g) baking powder", "2 tablespoon (21g) vegetable oil", "1 tsp (4g) vanilla extract, optional", "Butter to finish", "Maple syrup to finish", "1 cup (240ml) buttermilk", "1 egg", "1 egg yolk", "2 cups (300g) all-purpose flour", "\u00bd cup (120ml) milk", "\u00bd cup (95g) fresh blueberries", "1/4 cup (62g) granulated sugar", "1 teaspoon (4g) fine sea salt", "1/4 teaspoon (1g) baking soda", "1 tablespoon (12g) baking powder", "2 tablespoon (21g) vegetable oil", "1 tsp (4g) vanilla extract, optional", "Whipped cream to serve", "2 \u00bd cups (285g) fresh blueberries", "1 cup (200g) granulated white sugar", "\u00bd cup (120ml) water", "\u00bc cup (60ml) maple syrup", "1 tablespoon (17g) light corn syrup", "3 tablespoons (55g) tablespoon lemon juice", "Whipped Cream:", "1 cup (240ml) heavy whipping cream", "2 tablespoons (14g) powdered sugar", "1 tablespoon (15g) creme fraiche, optional", "1 cup (240ml) buttermilk", "1 egg", "1 egg yolk", "2 cups (300g) all-purpose flour", "\u00bd cup (120ml) whole milk", "1/4 cup (62g) granulated sugar", "1 teaspoon (4g) fine sea salt", "1/4 teaspoon (1g) baking soda", "1 tablespoon (12g) baking powder", "2 tablespoon (21g) vegetable oil", "1 tsp (4g) vanilla extract, optional", "Whipped cream to serve", "1 cup (225g) brown sugar", "1 Tbsp (14g) muscovado sugar", "\u00bd cup (120ml) water", "\u00bd cup (120ml) maple syrup", "2.5 Tbsp (17g) cinnamon", "1/3 cup (84g) butter", "1 teaspoon (7g) fine sea salt", "4 oz (115g) cream cheese", "3/4 cups (90g) powdered sugar", "1 vanilla bean", "2 tablespoons (35g) whole milk"], "instructions": "Into a large bowl, add all purpose flour, sugar, baking soda, baking powder, salt. Whisk together until fully combined.\nThen, to the mixing bowl, add buttermilk, whole milk, egg, egg yolk, and vegetable oil. Whisk until homogeneous, being careful not to overmix. Optionally, vanilla extract can be added for an extra pop of flavor.\nPreheat a griddle, or non-stick skillet, over medium heat. Once to temperature, lightly spray with cooking spray.\nOnce hot, spoon \u00bc\u201d dollops of the pancake batter. Cook for 2-3 minutes or until golden brown on the bottom.\nFlip and cook for an additional 2-3 minutes. Repeat with the rest of the batter.\nTop with butter and drizzle with maple syrup to serve.\nIn a small saucepan combine blueberries, sugar, maple syrup, corn syrup, and, water and bring to a boil over medium-low heat. Once boiling, bring mixture to a simmer and cook for 6-8 minutes, stirring occasionally.\nCut heat and pass through a fine mesh strainer into a small mixing bowl, then fold in a \u00bd cup of fresh blueberries. Let cool at room temperature\nAdd heavy cream to a medium sized bowl and whisk aggressively until it begins to thicken.\nAdd powdered sugar, and continue to whisk until soft peaks have formed.\nOptionally, creme fraiche can be folded in at the end for extra flavor.\nInto a large bowl, add all purpose flour, sugar, baking soda, baking powder, and salt. Whisk together until fully combined.\nThen, to the mixing bowl, add buttermilk, whole milk, egg, egg yolk, and vegetable oil. Whisk until homogeneous, being careful not to overmix. Optionally, vanilla extract can be added for an extra pop of flavor.\nOnce homogenized, fold in fresh blueberries.\nPreheat a griddle, or non-stick skillet over medium heat. Once to temperature, lightly spray with cooking spray.\nOnce hot, spoon \u00bc\u201d dollops of the pancake batter. Cook for 2-3 minutes or until golden brown on the bottom.\nFlip and cook for an additional 2-3 minutes. Repeat with the rest of the batter.\nTop with blueberry sauce and a dollop of whipped cream.\nIn a small saucepan add brown sugar, muscovado sugar, and water. Slowly heat the mixture to dissolve.\nOnce dissolved, add maple syrup and bring to a boil.\nHeat mixture to 230\u00b0F (110\u00b0C) and cut heat. Stir in cinnamon until fully combined.\nAdd cold butter to the cinnamon mixture and stir to mount butter into the spread. Season with salt to finish.\nMove to heat proof storage container and allow to cool at room temperature\nTo a medium sized bowl add cream cheese. Using an electric beater, mix the cream cheese until softened and spreadable.\nBegin to add the powdered sugar one half at a time. Mix until fully combined.\nBeat in whole milk until mixture is smooth\nAdd 1 scraped vanilla bean, and fold into mixture using a rubber spatula until evenly distributed.\nInto a large bowl, add all purpose flour, sugar, baking soda, baking powder, salt. Whisk together until fully combined.\nThen, to the mixing bowl, add buttermilk, whole milk, egg, egg yolk, and vegetable oil. Whisk until homogeneous, being careful not to overmix. Optionally, vanilla extract can be added for an extra pop of flavor.\nPreheat a griddle, or non-stick skillet over medium heat. Once to temperature, lightly spray with cooking spray.\nOnce hot spoon \u00bc\u201d dollops of the pancake batter. Cook for 2-3 minutes or until golden brown on the bottom.\nFlip and cook for an additional 2-3 minutes. Repeat with the rest of the batter.\nTo assemble, place a pancake on a large round plate then spread cinnamon sauce onto the pancake. Repeat this step until your stack is 4 pancakes tall. Spread your top pancake with the cinnamon sauce, then drizzle with your cream cheese glaze.", "instructions_list": ["Into a large bowl, add all purpose flour, sugar, baking soda, baking powder, salt. Whisk together until fully combined.", "Then, to the mixing bowl, add buttermilk, whole milk, egg, egg yolk, and vegetable oil. Whisk until homogeneous, being careful not to overmix. Optionally, vanilla extract can be added for an extra pop of flavor.", "Preheat a griddle, or non-stick skillet, over medium heat. Once to temperature, lightly spray with cooking spray.", "Once hot, spoon \u00bc\u201d dollops of the pancake batter. Cook for 2-3 minutes or until golden brown on the bottom.", "Flip and cook for an additional 2-3 minutes. Repeat with the rest of the batter.", "Top with butter and drizzle with maple syrup to serve.", "In a small saucepan combine blueberries, sugar, maple syrup, corn syrup, and, water and bring to a boil over medium-low heat. Once boiling, bring mixture to a simmer and cook for 6-8 minutes, stirring occasionally.", "Cut heat and pass through a fine mesh strainer into a small mixing bowl, then fold in a \u00bd cup of fresh blueberries. Let cool at room temperature", "Add heavy cream to a medium sized bowl and whisk aggressively until it begins to thicken.", "Add powdered sugar, and continue to whisk until soft peaks have formed.", "Optionally, creme fraiche can be folded in at the end for extra flavor.", "Into a large bowl, add all purpose flour, sugar, baking soda, baking powder, and salt. Whisk together until fully combined.", "Then, to the mixing bowl, add buttermilk, whole milk, egg, egg yolk, and vegetable oil. Whisk until homogeneous, being careful not to overmix. Optionally, vanilla extract can be added for an extra pop of flavor.", "Once homogenized, fold in fresh blueberries.", "Preheat a griddle, or non-stick skillet over medium heat. Once to temperature, lightly spray with cooking spray.", "Once hot, spoon \u00bc\u201d dollops of the pancake batter. Cook for 2-3 minutes or until golden brown on the bottom.", "Flip and cook for an additional 2-3 minutes. Repeat with the rest of the batter.", "Top with blueberry sauce and a dollop of whipped cream.", "In a small saucepan add brown sugar, muscovado sugar, and water. Slowly heat the mixture to dissolve.", "Once dissolved, add maple syrup and bring to a boil.", "Heat mixture to 230\u00b0F (110\u00b0C) and cut heat. Stir in cinnamon until fully combined.", "Add cold butter to the cinnamon mixture and stir to mount butter into the spread. Season with salt to finish.", "Move to heat proof storage container and allow to cool at room temperature", "To a medium sized bowl add cream cheese. Using an electric beater, mix the cream cheese until softened and spreadable.", "Begin to add the powdered sugar one half at a time. Mix until fully combined.", "Beat in whole milk until mixture is smooth", "Add 1 scraped vanilla bean, and fold into mixture using a rubber spatula until evenly distributed.", "Into a large bowl, add all purpose flour, sugar, baking soda, baking powder, salt. Whisk together until fully combined.", "Then, to the mixing bowl, add buttermilk, whole milk, egg, egg yolk, and vegetable oil. Whisk until homogeneous, being careful not to overmix. Optionally, vanilla extract can be added for an extra pop of flavor.", "Preheat a griddle, or non-stick skillet over medium heat. Once to temperature, lightly spray with cooking spray.", "Once hot spoon \u00bc\u201d dollops of the pancake batter. Cook for 2-3 minutes or until golden brown on the bottom.", "Flip and cook for an additional 2-3 minutes. Repeat with the rest of the batter.", "To assemble, place a pancake on a large round plate then spread cinnamon sauce onto the pancake. Repeat this step until your stack is 4 pancakes tall. Spread your top pancake with the cinnamon sauce, then drizzle with your cream cheese glaze."], "language": "en", "nutrients": {}, "site_name": "Joshua Weissman", "title": "Making IHOP Pancakes at Home | But Better"}

...