ethyca / fides

The Privacy Engineering & Compliance Framework
https://ethyca.com/docs
Apache License 2.0
360 stars 72 forks source link

LA-104: Integrate access request test against BigQuery Enterprise in CI #5504

Closed eastandwestwind closed 3 days ago

eastandwestwind commented 5 days ago

Closes https://ethyca.atlassian.net/browse/LA-104

Description Of Changes

This PR adds the first integration test where we make an access request against real-world data, as opposed to generated / fake data. This approach runs parallel but is distinctly different than our initiative to test big data, as in many rows, columns, tables, etc.

To test big data, we've adopted the Stackoverflow public dataset, where we'll have a solid foundation upon which to build future complex tables / data for BigQuery.

Code Changes

Steps to Confirm

  1. Confirm all tests pass in CI
  2. To manually run, ensure you have BIGQUERY_ENTERPRISE_KEYFILE_CREDS and BIGQUERY_ENTERPRISE_DATASET in your ENV file.
  3. Run nox -s dev -- shell
  4. Run pytest tests/ops/service/privacy_request/test_bigquery_enterprise_privacy_request.py
  5. Confirm test passes

Pre-Merge Checklist

vercel[bot] commented 5 days ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment | Name | Status | Preview | Comments | Updated (UTC) | | :--- | :----- | :------ | :------- | :------ | | **fides-plus-nightly** | ⬜️ Ignored ([Inspect](https://vercel.com/ethyca/fides-plus-nightly/EDesdHLDskk9uJBE2KYHTivjCvpb)) | [Visit Preview](https://fides-plus-nightly-git-la-104-ethyca.vercel.app) | | Nov 20, 2024 3:18pm |
cypress[bot] commented 5 days ago

fides    Run #11103

Run Properties:  status check passed Passed #11103  •  git commit 9c2e0a6147 ℹ️: Merge 55d25b123f924c0aed85e76323d3cc0c8d125f33 into 30c44648063eb2e6dd6d761bfb2b...
Project fides
Branch Review refs/pull/5504/merge
Run status status check passed Passed #11103
Run duration 00m 39s
Commit git commit 9c2e0a6147 ℹ️: Merge 55d25b123f924c0aed85e76323d3cc0c8d125f33 into 30c44648063eb2e6dd6d761bfb2b...
Committer Catherine Smith
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎
codecov[bot] commented 5 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 85.18%. Comparing base (d14d031) to head (66480d8). Report is 6 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #5504 +/- ## ========================================== - Coverage 85.20% 85.18% -0.03% ========================================== Files 386 387 +1 Lines 24247 24265 +18 Branches 2644 2646 +2 ========================================== + Hits 20660 20669 +9 - Misses 3033 3042 +9 Partials 554 554 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


🚨 Try these New Features:

cypress[bot] commented 3 days ago

fides    Run #11110

Run Properties:  status check passed Passed #11110  •  git commit 48dc19916f: LA-104: Integrate access request test against BigQuery Enterprise in CI (#5504)
Project fides
Branch Review main
Run status status check passed Passed #11110
Run duration 00m 40s
Commit git commit 48dc19916f: LA-104: Integrate access request test against BigQuery Enterprise in CI (#5504)
Committer Catherine Smith
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎