search

RSE-Sheffield / sheffield-covid-19

Scrape COVID-19 data from the University website
1 stars 8 forks source link

There are no tests #9

Closed drj11 closed 3 years ago

drj11 commented 3 years ago

at least one test would be to take a snapshot copy of the webpage, and then run the ingest code over it, and comparing the results against a known-good exemplar.

To start with this would involve refactoring the existing code a bit, so that it doesn't just print() its result.

muneerahp commented 3 years ago

Happy to pick this up, assuming you're happy with the fact that my python testing knowledge is currently somewhat slim.

drj11 commented 3 years ago

I've assigned you. I would try pytest if you don't have any other favourite testing frameworks.

muneerahp commented 3 years ago

Starting with pytest 6.1.1, structure /tests/test_ingest.py and the test suggested.

Options to get this more suitable to run the kind of test suggested include:

  1. Moving the whole extract, validate and transform block from main into a single method which given html returns the data in the format you want. Then creatingtest_<new_method>()
  2. Getting main to return data, and then creating test_main()
  3. Your suggestion here

I'm running with 1 for now (returning things from main seems shifty), suggest alternatives if any of the above sounds suspect.

drj11 commented 3 years ago

yes, approach 1 sounds good for now. Do feel free to create a PR early on and mark it as draft or WIP (Work In Progress).

muneerahp commented 3 years ago

21

drj11 commented 3 years ago

Since the description is no longer technically true, closing in favour of #25