mozilla / bigquery-etl

Bigquery ETL
https://mozilla.github.io/bigquery-etl
Mozilla Public License 2.0
241 stars 98 forks source link

add print statement to see why job might be failing #5853

Closed Marlene-M-Hirose closed 4 days ago

Marlene-M-Hirose commented 4 days ago

DAG is failing on the first run, added a test statement to see what the result is

Checklist for reviewer:

For modifications to schemas in restricted namespaces (see CODEOWNERS):

┆Issue is synchronized with this Jira Task

dataops-ci-bot commented 4 days ago

Integration report for "Merge branch 'main' into deng2631_add_test_print_statement_for_app_conversions_query"

sql.diff

Click to expand! ```diff diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/microsoft_derived/app_conversions_v1/query.py /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/microsoft_derived/app_conversions_v1/query.py --- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/microsoft_derived/app_conversions_v1/query.py 2024-06-27 02:56:39.000000000 +0000 +++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/microsoft_derived/app_conversions_v1/query.py 2024-06-27 02:56:33.000000000 +0000 @@ -214,6 +214,7 @@ json_none_string = change_null_to_string(json_file.text) # Convert the string to a dictionary query_export = eval(json_none_string) + print("This is query_export: ", query_export) if query_export["Value"]: # This section writes the tmp json data into a temp CSV file which will then be put into a BigQuery table microsoft_store_data = clean_json(query_export, date) ```

Link to full diff