search

hhursev / recipe-scrapers

Python package for scraping recipes data
MIT License
1.61k stars 505 forks source link

Added foodnetwork.com back #1026

Closed jlucaspains closed 3 months ago

jlucaspains commented 3 months ago

This PR fixes #1010

As described in the issue, this commit https://github.com/hhursev/recipe-scrapers/commit/4fef338670142c3b4563f902ded1ade1f8001d0f#diff-5d1c0cbbdbecea7561f2fa87a9ecdd8bb896262f71727989caa4cf140b54cdeb changed the URL for foodnetwork from foodnetwork.com to foodnetwork.co.uk. Additionally, foodntwork.com now is blocking requests from "old" user-agents and this PR updates the user-agent header.

I left foodnetwork.co.uk as the "default" domain and added the capability to also reference foodnetwork.com the same way as hellofresh scraper supports multiple domains. Tests are passing and I tried the following URLs locally and they both work as expected:

scraper = scrape_me('https://www.foodnetwork.com/recipes/alton-brown/shepherds-pie-recipe2-1942900', wild_mode=True)
scraper = scrape_me('https://foodnetwork.co.uk/recipes/tuscan-mushrooms-5389', wild_mode=True)
jayaddison commented 2 months ago

@jlucaspains this has been included and released in v14.56.0 (and also the pre-release v15.0.0-rc3) of recipe-scrapers on PyPi.