search

hhursev / recipe-scrapers

Python package for scraping recipes data
MIT License
1.7k stars 521 forks source link

Fix monsieurcuisine #602 #1014

Closed Keyruu closed 6 months ago

Keyruu commented 7 months ago

I'm not a python pro but I hope this code is good enough. This fixes the Monsieur Cuisine scraper for the new website. I use their API as mentioned in #602. Mind the Device-Type header, without this header the API won't return names for the ingredients for whatever reasons. Resolves #602

Keyruu commented 7 months ago

Like this?

jayaddison commented 6 months ago

Like this?

Almost, yep - the .testjson file you've added looks good, but the unit tests aren't reading from it yet. To make them do that, we'll need to add a legacy-style unit test.

There are a few examples of those in the tests/legacy directory here: https://github.com/hhursev/recipe-scrapers/tree/4da3a6493cebe91c85378a2307645006b4fb17a9/tests/legacy

I'd recommend test_coopse.py as one that should be relatively similar to this case, and could be a good option to copy from.

Legacy tests are used for the (relatively infrequent) cases like this where a scraper makes a second HTTP request after loading the initial HTML page. To test those, we configure the expected requests in the test case, and provide test responses.

Keyruu commented 6 months ago

Okay I added the legacy test. I really hope everything is done now...

jayaddison commented 6 months ago

@Keyruu thank you, this looks pretty good. Please could you edit the pull request description on GitHub to include the text Resolves #602? That will automatically close that bugreport if+when this is merged.

Keyruu commented 6 months ago

I have done everything you asked for. Can this be merged now?

jayaddison commented 6 months ago

I have done everything you asked for. Can this be merged now?

Soon, yep - there's an improvement related to the Accept-Language header that I'm thinking about; if I don't get it working soon, then I'll approve+merge without it, but I think we can clean up the code further.

Keyruu commented 6 months ago

Just added the expected headers in the test.

jayaddison commented 5 months ago

@Keyruu this has been included and released in v14.56.0 (and also the pre-release v15.0.0-rc3) of recipe-scrapers on PyPi.

Keyruu commented 5 months ago

Awesome thanks.