search

everypolitician / scraper_test

Data-driven scraper tests for Scraped
0 stars 0 forks source link

WIP: Target subset of data #16

Closed ondenman closed 7 years ago

ondenman commented 7 years ago

Fixes: https://github.com/everypolitician/scraper_test/issues/13

Sometimes, it may be preferable to test only a subset of the data returned by a scraper class.

With this commit, the YAML test file can contain a path field that targets a subset of data:

For example:

:url: http://majles.marsad.tn/2014/fr/assemblee
:path: members/id=34
:to_h: ...

The above looks for a hash with an :id of 34 within the :members array.

This is a step towards adding data test for scrapers with individual member pages: https://github.com/everypolitician/everypolitician/issues/600

Note: ae9bcad and fc1f9cb are refactors -- included to ensure the PR passes rubocop -- and need to be moved to their own PR.

ondenman commented 7 years ago

Closing in favour of: https://github.com/everypolitician/scraper_test/pull/22