search

hhursev / recipe-scrapers

Python package for scraping recipes data
MIT License
1.68k stars 520 forks source link

Add scraper for coop.se #929

Closed henrikolsson closed 9 months ago

henrikolsson commented 10 months ago

Adds a scraper for https://www.coop.se/recept/

This site is already listed in README.md but was removed in https://github.com/hhursev/recipe-scrapers/commit/4fef338670142c3b4563f902ded1ade1f8001d0f. Probably due to not working.

henrikolsson commented 10 months ago

Thanks for the review, I don't code Python often so it's easy to miss idiomatic conventions!

jayaddison commented 10 months ago

A few small lint errors and test expectations to fix up here @henrikolsson - hopefully all fairly straightforward. I'll take another look through the code again soon, thanks for applying those suggestions.

jayaddison commented 9 months ago

Please note: we've made a large-ish change to the format of the tests that affects this pull request: in short, the Python code that was used for most scraper tests has been replaced by a JSON file format that contain the expected values for each field (see #944).

That should be easier to manage -- and hopefully easier to developer scrapers with! -- long-term, although it does create some breaking changes for work-in-progress like here.

I'm going to investigate what's required to migrate this pull request to use the JSON format, and may fix up the lint/test issues at the same time.

jayaddison commented 9 months ago

Ugh, I've made a (large) mistake here: I forgot that this scraper makes multiple requests, and that the .testjson files are responses from API requests.

That implies that we should in fact keep the Python test case approach here -- we can't currently use the data-driven JSON expectation approach for scrapers that make multiple HTTP requests when they run.