Open barbarahui opened 8 months ago
The framework uses pytest. From pytest's perspective, there is only one test right now, called test_mappers
(defined in metadata_mapper/test/test_mapper.py
). This single test comprises:
--mapper
or --mappers
argto_UCLDC()
on each mapper's record class with fixture data as input. If any exceptions are raised, pytest reports this failure. The code uses pytest.assume to override the default halting behavior and continue iterating over any remaining mappers.Problem 1: we don't get output per mapper when running pytest
. This is because the default behavior for pytest is to list the tests that it "collects" (in this case, a single test) and then suppress any output unless the test fails. So for example, this is what we get when we run pytest for contentdm and csudh mappers:
$ pytest metadata_mapper/test/ --mapper=contentdm,csudh
================================================= test session starts ==================================================
platform darwin -- Python 3.9.0, pytest-7.4.4, pluggy-1.3.0
rootdir: /Users/bhui/dev/rikolti
plugins: Faker-22.5.0, assume-2.4.3, requests-mock-1.11.0
collected 1 item
metadata_mapper/test/test_mapper.py . [100%]
================================================== 1 passed in 1.26s ===================================================
Pytest simply reports that one test passed. What we need is some output on which mappers were tested. I think this can be accomplished by tweaking pytest's output capture behavior and/or logging behavior.
Problem 2: we don't get any error output when running pytest
for a non-existent mapper, e.g.:
$ pytest metadata_mapper/test/ --mapper=foobar
================================================= test session starts ==================================================
platform darwin -- Python 3.9.0, pytest-7.4.4, pluggy-1.3.0
rootdir: /Users/bhui/dev/rikolti
plugins: Faker-22.5.0, assume-2.4.3, requests-mock-1.11.0
collected 1 item
metadata_mapper/test/test_mapper.py . [100%]
================================================== 1 passed in 0.81s ==================================================
Again, I think we can tweak the capture and/or logging behavior to fix this.
Nice to have: it might be nice to get output on which mappers are missing helpers
Some things in Tim's PR that would be good to tidy up:
We looked at Tim's mapper unit testing PR 530 and it still doesn't print output per mapper when run. Otherwise the PR looks fine.
We're waiting to see how many CiC hours we have left once Lucas finishes the Tind mapper work. If there is still time on the books, we'll have Tim do the work to add output per mapper. Otherwise, CDL will do the work at some point.
We decided that mapper unit testing isn't critical for MVP, so can potentially wait until after cutover. However, it will definitely be useful in the future when doing mapper development.
See Barbara's notes below on how to approach this work if CDL ends up being the one to do it.