hhursev / recipe-scrapers

Python package for scraping recipes data
MIT License
1.66k stars 514 forks source link

Diet Doctor scraper #959

Closed sevmonster closed 8 months ago

sevmonster commented 8 months ago

I tried importing into Tandoor Recipes which uses this project, and no steps were imported.

Examples: https://www.dietdoctor.com/recipes/lemony-butter-baked-salmon-with-cauliflower-mash https://www.dietdoctor.com/recipes/fat-head-pizza https://www.dietdoctor.com/recipes/classic-keto-hamburger

https://github.com/TandoorRecipes/recipes/issues/2780

hhursev commented 8 months ago

Should be good in version 14.53.0 . Feel free to reopen if that's not the case

jknndy commented 8 months ago

wild_mode covers all the fields except instructions but I just opened #961 which adds support for that field.

sevmonster commented 8 months ago

It still seems to not import instructions in Tandoor Recipies latest beta, with recipe-scrapers 14.53. All ingredients are put on a single step with no instructions.

Screenshot_20231226-002528

I see that the module seems to be returning correct info, so I am not sure where this should be reported.

# scraper — not defined
>>> from recipe_scrapers import scrape_me
>>> scraper = scrape_me('https://www.dietdoctor.com/recipes/classic-keto-hamburger')
    Traceback (most recent call last):
  File "/home/sev/src/recipe-scraper/lib/python3.11/site-packages/recipe_scrapers/__init__.py", line 630, in scrape_me
    scraper = SCRAPERS[host_name]
              ~~~~~~~~^^^^^^^^^^^
KeyError: 'dietdoctor.com'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sev/src/recipe-scraper/lib/python3.11/site-packages/recipe_scrapers/__init__.py", line 633, in scrape_me
    raise WebsiteNotImplementedError(host_name)
recipe_scrapers._exceptions.WebsiteNotImplementedError: recipe-scrapers exception: Website (dietdoctor.com) not supported.

# wild_mode
>>> scraper = scrape_me('https://www.dietdoctor.com/recipes/classic-keto-hamburger', wild_mode=True)
>>> scraper.ingredients()
['1 1/4 cups almond flour', '5 tbsp ground psyllium husk powder', '2 tsp baking powder', '1 tsp sea salt', '1 1/4 cups water', '2 tsp white wine vinegar', '3 egg whites', '1 tbsp sesame seeds', '1 3/4 lbs ground beef', '2 tbsp butter', 'salt', '2 oz. lettuce', '1 tomato', '1 red onion', '1/2 cup mayonnaise', '5 oz. bacon', 'pepper']
>>> scraper.instructions_list()
['Preheat the oven to 350°F (175°C).', 'Start by making the hamburger buns. Mix together the dry ingredients in a bowl.', "Bring water to a boil. Add the water, the vinegar and the egg whites to the bowl while beating with a hand mixer for about 30 seconds. Don't overmix the dough; the consistency should resemble Play-Doh.", 'With moist hands, form the dough into individual pieces of bread, one for each serving. Sprinkle sesame seeds on top. Be sure to leave enough room on the baking sheet for the buns to double in size.', "Bake on lower rack in oven for 50-60 minutes. They're done when you hear a hollow sound when tapping the bottom of the buns.", 'Fry the bacon slices over medium heat.', 'Form the ground beef into individual hamburgers and either grill or fry. Season with salt and pepper when the hamburgers are almost done.', 'Split each roll in half and spread a generous amount of mayonnaise on each half.', 'Build your hamburger to your taste.', 'Pair with a side of coleslaw for added crunch!']

# html
>>> import requests
>>> from recipe_scrapers import scrape_html
>>> url = 'https://www.dietdoctor.com/recipes/classic-keto-hamburger'
>>> html = requests.get(url).content
>>> scraper = scrape_html(html=html, org_url=url)                                       
>>> scraper.instructions_list()
['Preheat the oven to 350°F (175°C).', 'Start by making the hamburger buns. Mix together the dry ingredients in a bowl.', "Bring water to a boil. Add the water, the vinegar and the egg whites to the bowl while beating with a hand mixer for about 30 seconds. Don't overmix the dough; the consistency should resemble Play-Doh.", 'With moist hands, form the dough into individual pieces of bread, one for each serving. Sprinkle sesame seeds on top. Be sure to leave enough room on the baking sheet for the buns to double in size.', "Bake on lower rack in oven for 50-60 minutes. They're done when you hear a hollow sound when tapping the bottom of the buns.", 'Fry the bacon slices over medium heat.', 'Form the ground beef into individual hamburgers and either grill or fry. Season with salt and pepper when the hamburgers are almost done.', 'Split each roll in half and spread a generous amount of mayonnaise on each half.', 'Build your hamburger to your taste.', 'Pair with a side of coleslaw for added crunch!']
>>> scraper.ingredients()
['1 1/4 cups almond flour', '5 tbsp ground psyllium husk powder', '2 tsp baking powder', '1 tsp sea salt', '1 1/4 cups water', '2 tsp white wine vinegar', '3 egg whites', '1 tbsp sesame seeds', '1 3/4 lbs ground beef', '2 tbsp butter', 'salt', '2 oz. lettuce', '1 tomato', '1 red onion', '1/2 cup mayonnaise', '5 oz. bacon', 'pepper']