OpenMined / PySyft

Perform data science on data that remains in someone else's server
https://www.openmined.org/
Apache License 2.0
9.49k stars 2k forks source link

Create mock bigquery client so bigquery API endpoints can be run without a real API #9196

Closed khoaguin closed 2 months ago

khoaguin commented 2 months ago

Issue: https://github.com/OpenMined/Heartbeat/issues/1762

review-notebook-app[bot] commented 2 months ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

khoaguin commented 2 months ago

image

khoaguin commented 2 months ago

Usage: we change the settings's credentials key to be a dict with value of "mock" and the expected output (can be "success", "bigquery_error", "timeout", "error")

@sy.api_endpoint_method(
    settings={
        "credentials": {"mock": "timeout"},  # the value can be: "success", "bigquery_error", "timeout",  "error"
        "region": secrets["region_bigquery"],
        "project_id": secrets["project_id"],
    }
)
def private_query_function(
    context,
    sql_query: str,
) -> str:
    if "mock" in context.settings["credentials"]:
        # syft absolute
        from syft.util.util import MockBigQuery as bigquery

        scoped_credentials = bigquery.mock_credentials(
            mock_result=context.settings["credentials"]["mock"]
        )
    else:
        # third party
        from google.cloud import bigquery
        from google.oauth2 import service_account

        # Auth for Bigquer based on the workload identity
        credentials = service_account.Credentials.from_service_account_info(
            context.settings["credentials"]
        )
        scoped_credentials = credentials.with_scopes(
            ["https://www.googleapis.com/auth/cloud-platform"]
        )