freelawproject / juriscraper

An API to scrape American court websites for metadata.
https://free.law/juriscraper/
BSD 2-Clause "Simplified" License
341 stars 98 forks source link

Testing documentation needs a clean up #749

Open johnhawkinson opened 8 months ago

johnhawkinson commented 8 months ago

There is so much wrong with the testing documentation, it's really annoying to figure out how to run tests if you're not super-steeped in this stuff.

tl;dr

Run all tests with

python -m unittest -v

Or, for an specific module:

python -m unittest -v tests.local.test_PacerParseCaseQueryTest

but actually

README.rst tells us to use tox as a test runner, but that fails:

(juriscraper) jhawk@lrr juriscraper % tox
py38: failed with pass_env values cannot contain whitespace, use comma to have multiple values in a single line, invalid values found 'PACER_USERNAME PACER_PASSWORD'

This seems to be a bad tox.ini file.

The actual test runner we do use in the github tests claims to be deprecated, but we use it anyhow (python setup.py test)and it seems to do much the same thing as python -m unittest -v.

The example in README.rst is python3 -m unittest -v tests.local.test_DateTest.DateTest.test_various_date_extractions but that gives:

======================================================================
ERROR: test_various_date_extractions (unittest.loader._FailedTest.test_various_date_extractions)
----------------------------------------------------------------------
AttributeError: type object 'DateTest' has no attribute 'test_various_date_extractions'

it looks like you need to prefix tests.local, as: python3 setup.py test -s tests.local.test_DocketParseTest.DocketAnonymizeTest.test_anonymize_bankruptcy

anyhow

Somebody who understands how this should work should fix the documentation to be accurate and usable and consistent with what the Github tests do (that is three work items).

I am trying to actually get timely stuff done, so this is off my plate.

mlissner commented 8 months ago

Thanks for filing. We'll get this cleaned up.

honeykjoule commented 8 months ago

ok this issue explains a lot. +1 confusing for new contributors

flooie commented 7 months ago

I also agree and I think we need a way to pass in just the specific court you want to test inside of tests. I shouldn't have to run 200 state courts just to generate or test Wyoming.