search

PatchPorting / patch-finder

A tool to find security patches from across the web.
GNU General Public License v2.0
4 stars 6 forks source link

Testing xpaths #9

Open jajajasalu2 opened 5 years ago

jajajasalu2 commented 5 years ago

Current unit tests use HTML files stored in tests/mocks. While using HTML mocks is helpful in testing the parsing logic, there is no way to check if a current online response is parsed correctly by the xpaths. Testing real time data would enable testing the xpaths in patchfinder.entrypoint.

A solution like https://github.com/kevin1024/vcrpy is optimal. vcrpy, unfortunately, does not currently support Scrapy's networking framework Twisted.

Thus, an efficient method to test if a current response is parsed properly is required.

jajajasalu2 commented 5 years ago

One possibility for this:

Use requests/urllib in test cases and create fake Scrapy responses from these responses. Decorate the test methods with vcrpy, so that response bodies can be taken from VCR cassettes later, and updated when necessary. This allows to "switch the mock on and off".