great-expectations / great_expectations

Always know what to expect from your data.
https://docs.greatexpectations.io/
Apache License 2.0
9.88k stars 1.53k forks source link

datasource.add_query_asset not respecting create_temp_table equals false #8147

Closed Chr96er closed 1 month ago

Chr96er commented 1 year ago

This issue is analogue to https://github.com/great-expectations/great_expectations/issues/6064.

Describe the bug When using datasource.add_query_asset, the create_temp_table=false is not respected.

Reproducible example:

import great_expectations as gx

context = gx.get_context()

connection_string = """bigquery://bigquery-public-data/samples"""

datasource = context.sources.add_or_update_sql(
    name="github_timeline", connection_string=connection_string, create_temp_table=False
)

table_asset = datasource.add_query_asset(
        name="github_timeline", 
        query="SELECT * FROM `bigquery-public-data.samples.github_timeline`"
)

my_batch_request = table_asset.build_batch_request()

batches = table_asset.get_batch_list_from_batch_request(my_batch_request)

context.add_or_update_expectation_suite("github_timeline_suite")

validator = context.get_validator(
    batch_request=my_batch_request,
    expectation_suite_name="github_timeline_suite",
)

validator.expect_column_values_to_not_be_null(column = "repository_url")
NotFound: 404 GET https://bigquery.googleapis.com/bigquery/v2/projects/bigquery-public-data/datasets/samples/tables/great_expectations_sub_selection?prettyPrint=false: Not found: Table bigquery-public-data:samples.great_expectations_sub_selection

During handling of the above exception, another exception occurred:

NoSuchTableError                          Traceback (most recent call last)

Environment (please complete the following information):

Operating System: Ubuntu Great Expectations Version: 0.17.0 .

HaebichanGX commented 1 year ago

Hey @Chr96er thanks again for raising this issue! I've put it in our backlog for review.

Kilo59 commented 3 months ago

@Chr96er, I believe this was fixed a few version back. But I don't recall exactly which one.

molliemarie commented 1 month ago

Hello @Chr96er. With the upcoming launch of Great Expectations Core (GX 1.0), we are closing old issues posted regarding previous versions. Moving forward, we will focus our resources on supporting and improving GX Core (version 1.0 and beyond). If you find that an issue you previously reported still exists in GX Core, we encourage you to resubmit it against the new version. With more resources dedicated to community support, we aim to tackle new issues swiftly. For specific details on what is GX-supported vs community-supported, you can reference our integration and support policy.

To get started on your transition to GX Core, check out the GX Core quickstart (click “Full example code” tab to see a code example).

You can also join our upcoming community meeting on August 28th at 9am PT (noon ET / 4pm UTC) for a comprehensive rundown of everything GX Core, plus Q&A as time permits. Go to https://greatexpectations.io/meetup and click “follow calendar” to follow the GX community calendar.

Thank you for being part of the GX community and thank you for submitting this issue. We're excited about this new chapter and look forward to your feedback on GX Core. 🤗