censoredplanet / censoredplanet-analysis

Analysis of the CensoredPlanet data.
Apache License 2.0
14 stars 5 forks source link

add base data size sanity check #221

Closed ohnorobo closed 1 year ago

ohnorobo commented 1 year ago

Add an e2e test which checks that the base tables have the expected number of measurement ids.

We will likely want to expand this to:

Currently this can't be submitted since it fails (since dev is missing data). But once we get the base datasets fixed up we can add it in so it runs regularly.

tested: manually set the table to censoredplanet-analysisv1.agix.https_scan and saw that the test passed. tested: tests passed after fixing dev in https://github.com/censoredplanet/censoredplanet-analysis/issues/220#issuecomment-1486669350