hhursev / recipe-scrapers

Python package for scraping recipes data
MIT License
1.62k stars 508 forks source link

Run all tests for scraper even if an assertion fails #988

Closed strangetom closed 6 months ago

strangetom commented 6 months ago

Prompted by @jayaddison's recent comment on #944, this changes how the test function for each scraper is built.

Currently, if an assertion fails for a scraper, the test function exists immediately and doesn't run any further tests for that scraper. With this change, each test is now run within a unittest.subTest() context manager, so that all the tests will run even if an assertion fails.

As an example, I made two minor changes to the pickuplimes.json file to cause two of the tests to fail.

Output before this change

$ python -m unittest -k pickuplimes
F
======================================================================
FAIL: tests/test_data/pickuplimes.com/pickuplimes.json (tests.RecipeTestCase.tests/test_data/pickuplimes.com/pickuplimes.json)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tom/Recipes/recipe-scrapers/tests/__init__.py", line 86, in test_func
    self.assertEqual(
AssertionError: 'Pick UpLimes' != 'Pick Up Limes'
- Pick UpLimes
+ Pick Up Limes
?        +
 : The actual value for .author() did not match the expected value.

----------------------------------------------------------------------
Ran 1 test in 0.091s

FAILED (failures=1)

Output after this change

$ python -m unittest -k pickuplimes
FF
======================================================================
FAIL: tests/test_data/pickuplimes.com/pickuplimes.json (tests.RecipeTestCase.tests/test_data/pickuplimes.com/pickuplimes.json) [author]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tom/Recipes/recipe-scrapers/tests/__init__.py", line 87, in test_func
    self.assertEqual(
AssertionError: 'Pick UpLimes' != 'Pick Up Limes'
- Pick UpLimes
+ Pick Up Limes
?        +
 : The actual value for .author() did not match the expected value.

======================================================================
FAIL: tests/test_data/pickuplimes.com/pickuplimes.json (tests.RecipeTestCase.tests/test_data/pickuplimes.com/pickuplimes.json) [cook_time]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tom/Recipes/recipe-scrapers/tests/__init__.py", line 105, in test_func
    self.assertEqual(
AssertionError: 12 != 10 : The actual value for .cook_time() did not match the expected value.

----------------------------------------------------------------------
Ran 1 test in 0.182s

FAILED (failures=2)
jayaddison commented 6 months ago

A suggestion: could we also make the ingredient grouping test a subtest?