OneZoom / tree-build

Scripts for assembling the tree, metadata and downstream data products such as popularity and popular images
MIT License
1 stars 2 forks source link

Tests partially broken when using `--real-apis` #71

Closed hyanwong closed 2 months ago

hyanwong commented 3 months ago

I implemented a "--real-apis" flag to the test suite so that we can use the real wiki & azure APIs for testing if we want. This has to use the correct QID, but can fake a negative OTT. All the test functions now save their images in a tmp directory, so we shouldn't risk stomping on existing data. We can't run all the parts of all the tests, because we can't guarantee the API replies (e.g. will we always get the same vernacular). But we can do some checking, e.g. the vernacular API call for Q140 should always generate one row that contains "Lion", and we can probably be guaranteed that there will be a picture (although we can't guarantee what the picture is)

However,

  1. It's only partially done. The functions in test_get_wiki_images.py::TestAPI do not account for this flag: it's only TestCLI that does.
  2. Even in the TestCLI functions, we get the failure below, which I haven't had time to investigate
E       assert (-772, 2, 61, 1234, 1) == (-772, 2, 62, 42000, 1)
E         
E         At index 2 diff: 61 != 62
E         Use -v to get more diff

tests/test_get_wiki_images.py:350: AssertionError
-------------------------------------------------------------- Captured stdout call --------------------------------------------------------------
((-772, 140, 'Panthera leo'),) SELECT ott,wikidata,name FROM ordered_leaves WHERE ott=%s; -772
--------------------------------------------------------------- Captured log call ----------------------------------------------------------------
WARNING  get_wiki_images.py:get_wiki_images.py:245 Unknown image 'SecondLionImage.jpg'
WARNING  get_wiki_images.py:get_wiki_images.py:350 Couldn't get license or artist for 'SecondLionImage.jpg;. Ignoring it.
============================================================ short test summary info =============================================================
FAILED tests/test_get_wiki_images.py::TestCLI::test_get_leaf_bespoke_image - assert (-772, 2, 61, 1234, 1) == (-772, 2, 62, 42000, 1)
davidebbo commented 2 months ago

It fails because we're calling verify_image_behavior with an image name of SecondLionImage.jpg, which doesn't actually exist in wikimedia.

hyanwong commented 2 months ago

Ah, of course! I guess we could make that any string we like, and we could choose something with is probably going to exist indefinitely on wikimedia commons, any anything springs to mind (any of the public domain "featured images" will probably continue existing on wikimedia commons).

davidebbo commented 2 months ago

I pushed a change to fix the test. There was an additional issue with not having a licence string in the test entry.

hyanwong commented 2 months ago

Thanks!

hyanwong commented 2 months ago

By the way, if you are pushing to the branch with the ruff linting, you can pre-commit install to get the linter to automatically run when you commit any changes. You might have done that already.

hyanwong commented 2 months ago

Thanks for fixing this @davidebbo: -real-apis all passes now.