airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.92k stars 4.09k forks source link

🐛 Source Google sheets: import fails at nomination for one sheet #9865

Closed nthbooth-feedr closed 1 year ago

nthbooth-feedr commented 2 years ago

Environment

Current Behavior

Most tables in the sheet sync with out issue but there is an issue by one July 2021 which results in the an error. I have worked arround this by stoping this one sheet syncing Logs are attached.

Expected Behavior

Sync to work for all sheets in the doc.

Logs

* logs-2906-2.txt

2022-01-27 17:15:54 normalization >   create  table
2022-01-27 17:15:54 normalization >     "feedr".customer_issue_tracker."july_2021__dbt_tmp"
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization >       compound sortkey(_airbyte_emitted_at)
2022-01-27 17:15:54 normalization >   as (
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization > with __dbt__cte__july_2021_ab1 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to parse JSON blob stored in a single column and extract into separated field columns as described by the JSON Schema
2022-01-27 17:15:54 normalization > -- depends_on: "feedr".customer_issue_tracker._airbyte_raw_july_2021
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Client', true) != '' then json_extract_path_text(_airbyte_data, 'Client', true) end as client,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Vendor', true) != '' then json_extract_path_text(_airbyte_data, 'Vendor', true) end as vendor,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Service', true) != '' then json_extract_path_text(_airbyte_data, 'Service', true) end as service,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Location', true) != '' then json_extract_path_text(_airbyte_data, 'Location', true) end as location,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Reporter', true) != '' then json_extract_path_text(_airbyte_data, 'Reporter', true) end as reporter,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'User email', true) != '' then json_extract_path_text(_airbyte_data, 'User email', true) end as "user email",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to IA?', true) != '' then json_extract_path_text(_airbyte_data, 'Added to IA?', true) end as "added to ia?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Complete by?', true) != '' then json_extract_path_text(_airbyte_data, 'Complete by?', true) end as "complete by?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Order number', true) != '' then json_extract_path_text(_airbyte_data, 'Order number', true) end as "order number",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Contacted by?', true) != '' then json_extract_path_text(_airbyte_data, 'Contacted by?', true) end as "contacted by?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Date of issue', true) != '' then json_extract_path_text(_airbyte_data, 'Date of issue', true) end as "date of issue",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Ops team resolved', true) != '' then json_extract_path_text(_airbyte_data, 'Ops team resolved', true) end as "ops team resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Customer impacting?', true) != '' then json_extract_path_text(_airbyte_data, 'Customer impacting?', true) end as "customer impacting?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Mins if late/ early', true) != '' then json_extract_path_text(_airbyte_data, 'Mins if late/ early', true) end as "mins if late/ early",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to tracker (ops)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to tracker (ops)', true) end as "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'AM Team Acknowledge/ Resolved', true) != '' then json_extract_path_text(_airbyte_data, 'AM Team Acknowledge/ Resolved', true) end as "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to invoice tracker - (AM)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to invoice tracker - (AM)', true) end as "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to invoice tracker (Vendor)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to invoice tracker (Vendor)', true) end as "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Vendor Team Acknowledge/ Resolved', true) != '' then json_extract_path_text(_airbyte_data, 'Vendor Team Acknowledge/ Resolved', true) end as "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Issue reason 2
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) != '' then json_extract_path_text(_airbyte_data, 'Issue reason 2
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) end as "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Issue reason 1 
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) != '' then json_extract_path_text(_airbyte_data, 'Issue reason 1 
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) end as "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Description of issue - including any immediate resolution (include item values)', true) != '' then json_extract_path_text(_airbyte_data, 'Description of issue - including any immediate resolution (include item values)', true) end as "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Further resolution notes from Ops, Vendor team, AM team, ops team (including details of charges)', true) != '' then json_extract_path_text(_airbyte_data, 'Further resolution notes from Ops, Vendor team, AM team, ops team (including details of charges)', true) end as "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at
2022-01-27 17:15:54 normalization > from "feedr".customer_issue_tracker._airbyte_raw_july_2021 as table_alias
2022-01-27 17:15:54 normalization > -- july_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > ),  __dbt__cte__july_2021_ab2 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to cast each column to its adequate SQL type converted from the JSON schema type
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__july_2021_ab1
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     cast(client as varchar) as client,
2022-01-27 17:15:54 normalization >     cast(vendor as varchar) as vendor,
2022-01-27 17:15:54 normalization >     cast(service as varchar) as service,
2022-01-27 17:15:54 normalization >     cast(location as varchar) as location,
2022-01-27 17:15:54 normalization >     cast(reporter as varchar) as reporter,
2022-01-27 17:15:54 normalization >     cast("user email" as varchar) as "user email",
2022-01-27 17:15:54 normalization >     cast("added to ia?" as varchar) as "added to ia?",
2022-01-27 17:15:54 normalization >     cast("complete by?" as varchar) as "complete by?",
2022-01-27 17:15:54 normalization >     cast("order number" as varchar) as "order number",
2022-01-27 17:15:54 normalization >     cast("contacted by?" as varchar) as "contacted by?",
2022-01-27 17:15:54 normalization >     cast("date of issue" as varchar) as "date of issue",
2022-01-27 17:15:54 normalization >     cast("ops team resolved" as varchar) as "ops team resolved",
2022-01-27 17:15:54 normalization >     cast("customer impacting?" as varchar) as "customer impacting?",
2022-01-27 17:15:54 normalization >     cast("mins if late/ early" as varchar) as "mins if late/ early",
2022-01-27 17:15:54 normalization >     cast("added to tracker (ops)" as varchar) as "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     cast("am team acknowledge/ resolved" as varchar) as "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     cast("added to invoice tracker - (am)" as varchar) as "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     cast("added to invoice tracker (vendor)" as varchar) as "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     cast("vendor team acknowledge/ resolved" as varchar) as "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     cast("issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar) as "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     cast("issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar) as "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     cast("description of issue - including any immediate resolution (include item values)" as varchar) as "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     cast("further resolution notes from ops, vendor team, am team, ops team (including details of charges)" as varchar) as "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at
2022-01-27 17:15:54 normalization > from __dbt__cte__july_2021_ab1
2022-01-27 17:15:54 normalization > -- july_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > ),  __dbt__cte__july_2021_ab3 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to build a hash column based on the values of this record
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__july_2021_ab2
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     md5(cast(coalesce(cast(client as varchar), '') || '-' || coalesce(cast(vendor as varchar), '') || '-' || coalesce(cast(service as varchar), '') || '-' || coalesce(cast(location as varchar), '') || '-' || coalesce(cast(reporter as varchar), '') || '-' || coalesce(cast("user email" as varchar), '') || '-' || coalesce(cast("added to ia?" as varchar), '') || '-' || coalesce(cast("complete by?" as varchar), '') || '-' || coalesce(cast("order number" as varchar), '') || '-' || coalesce(cast("contacted by?" as varchar), '') || '-' || coalesce(cast("date of issue" as varchar), '') || '-' || coalesce(cast("ops team resolved" as varchar), '') || '-' || coalesce(cast("customer impacting?" as varchar), '') || '-' || coalesce(cast("mins if late/ early" as varchar), '') || '-' || coalesce(cast("added to tracker (ops)" as varchar), '') || '-' || coalesce(cast("am team acknowledge/ resolved" as varchar), '') || '-' || coalesce(cast("added to invoice tracker - (am)" as varchar), '') || '-' || coalesce(cast("added to invoice tracker (vendor)" as varchar), '') || '-' || coalesce(cast("vendor team acknowledge/ resolved" as varchar), '') || '-' || coalesce(cast("issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar), '') || '-' || coalesce(cast("issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar), '') || '-' || coalesce(cast("description of issue - including any immediate resolution (include item values)" as varchar), '') || '-' || coalesce(cast("further resolution notes from ops, vendor team, am team, ops team (including details of charges)" as varchar), '') as varchar)) as _airbyte_july_2021_hashid,
2022-01-27 17:15:54 normalization >     tmp.*
2022-01-27 17:15:54 normalization > from __dbt__cte__july_2021_ab2 tmp
2022-01-27 17:15:54 normalization > -- july_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > )-- Final base SQL model
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__july_2021_ab3
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     client,
2022-01-27 17:15:54 normalization >     vendor,
2022-01-27 17:15:54 normalization >     service,
2022-01-27 17:15:54 normalization >     location,
2022-01-27 17:15:54 normalization >     reporter,
2022-01-27 17:15:54 normalization >     "user email",
2022-01-27 17:15:54 normalization >     "added to ia?",
2022-01-27 17:15:54 normalization >     "complete by?",
2022-01-27 17:15:54 normalization >     "order number",
2022-01-27 17:15:54 normalization >     "contacted by?",
2022-01-27 17:15:54 normalization >     "date of issue",
2022-01-27 17:15:54 normalization >     "ops team resolved",
2022-01-27 17:15:54 normalization >     "customer impacting?",
2022-01-27 17:15:54 normalization >     "mins if late/ early",
2022-01-27 17:15:54 normalization >     "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at,
2022-01-27 17:15:54 normalization >     _airbyte_july_2021_hashid
2022-01-27 17:15:54 normalization > from __dbt__cte__july_2021_ab3
2022-01-27 17:15:54 normalization > -- july_2021 from "feedr".customer_issue_tracker._airbyte_raw_july_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization >   );
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.554352 (Thread-8): SQL status: BEGIN in 0.22 seconds
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.554601 (Thread-8): Using redshift connection "model.airbyte_utils.october_2021".
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.554730 (Thread-8): On model.airbyte_utils.october_2021: /* {"app": "dbt", "dbt_version": "0.21.1", "profile_name": "normalize", "target_name": "prod", "node_id": "model.airbyte_utils.october_2021"} */
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization >   create  table
2022-01-27 17:15:54 normalization >     "feedr".customer_issue_tracker."october_2021__dbt_tmp"
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization >       compound sortkey(_airbyte_emitted_at)
2022-01-27 17:15:54 normalization >   as (
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization > with __dbt__cte__october_2021_ab1 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to parse JSON blob stored in a single column and extract into separated field columns as described by the JSON Schema
2022-01-27 17:15:54 normalization > -- depends_on: "feedr".customer_issue_tracker._airbyte_raw_october_2021
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Date', true) != '' then json_extract_path_text(_airbyte_data, 'Date', true) end as date,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Client', true) != '' then json_extract_path_text(_airbyte_data, 'Client', true) end as client,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Vendor', true) != '' then json_extract_path_text(_airbyte_data, 'Vendor', true) end as vendor,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Service', true) != '' then json_extract_path_text(_airbyte_data, 'Service', true) end as service,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Location', true) != '' then json_extract_path_text(_airbyte_data, 'Location', true) end as location,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Reporter', true) != '' then json_extract_path_text(_airbyte_data, 'Reporter', true) end as reporter,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'User email', true) != '' then json_extract_path_text(_airbyte_data, 'User email', true) end as "user email",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to IA?', true) != '' then json_extract_path_text(_airbyte_data, 'Added to IA?', true) end as "added to ia?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Order number', true) != '' then json_extract_path_text(_airbyte_data, 'Order number', true) end as "order number",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Contacted by?', true) != '' then json_extract_path_text(_airbyte_data, 'Contacted by?', true) end as "contacted by?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Ops team resolved', true) != '' then json_extract_path_text(_airbyte_data, 'Ops team resolved', true) end as "ops team resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Customer impacting?', true) != '' then json_extract_path_text(_airbyte_data, 'Customer impacting?', true) end as "customer impacting?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Mins if late/ early', true) != '' then json_extract_path_text(_airbyte_data, 'Mins if late/ early', true) end as "mins if late/ early",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to tracker (ops)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to tracker (ops)', true) end as "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'AM Team Acknowledge/ Resolved', true) != '' then json_extract_path_text(_airbyte_data, 'AM Team Acknowledge/ Resolved', true) end as "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to invoice tracker - (AM)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to invoice tracker - (AM)', true) end as "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to invoice tracker (Vendor)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to invoice tracker (Vendor)', true) end as "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Vendor Team Acknowledge/ Resolved', true) != '' then json_extract_path_text(_airbyte_data, 'Vendor Team Acknowledge/ Resolved', true) end as "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Issue reason 2
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) != '' then json_extract_path_text(_airbyte_data, 'Issue reason 2
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) end as "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Issue reason 1 
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) != '' then json_extract_path_text(_airbyte_data, 'Issue reason 1 
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) end as "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Description of issue - including any immediate resolution (include item values)', true) != '' then json_extract_path_text(_airbyte_data, 'Description of issue - including any immediate resolution (include item values)', true) end as "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Further resolution notes from Ops, Vendor team, AM team, ops team (including details of charges)', true) != '' then json_extract_path_text(_airbyte_data, 'Further resolution notes from Ops, Vendor team, AM team, ops team (including details of charges)', true) end as "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at
2022-01-27 17:15:54 normalization > from "feedr".customer_issue_tracker._airbyte_raw_october_2021 as table_alias
2022-01-27 17:15:54 normalization > -- october_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > ),  __dbt__cte__october_2021_ab2 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to cast each column to its adequate SQL type converted from the JSON schema type
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__october_2021_ab1
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     cast(date as varchar) as date,
2022-01-27 17:15:54 normalization >     cast(client as varchar) as client,
2022-01-27 17:15:54 normalization >     cast(vendor as varchar) as vendor,
2022-01-27 17:15:54 normalization >     cast(service as varchar) as service,
2022-01-27 17:15:54 normalization >     cast(location as varchar) as location,
2022-01-27 17:15:54 normalization >     cast(reporter as varchar) as reporter,
2022-01-27 17:15:54 normalization >     cast("user email" as varchar) as "user email",
2022-01-27 17:15:54 normalization >     cast("added to ia?" as varchar) as "added to ia?",
2022-01-27 17:15:54 normalization >     cast("order number" as varchar) as "order number",
2022-01-27 17:15:54 normalization >     cast("contacted by?" as varchar) as "contacted by?",
2022-01-27 17:15:54 normalization >     cast("ops team resolved" as varchar) as "ops team resolved",
2022-01-27 17:15:54 normalization >     cast("customer impacting?" as varchar) as "customer impacting?",
2022-01-27 17:15:54 normalization >     cast("mins if late/ early" as varchar) as "mins if late/ early",
2022-01-27 17:15:54 normalization >     cast("added to tracker (ops)" as varchar) as "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     cast("am team acknowledge/ resolved" as varchar) as "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     cast("added to invoice tracker - (am)" as varchar) as "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     cast("added to invoice tracker (vendor)" as varchar) as "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     cast("vendor team acknowledge/ resolved" as varchar) as "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     cast("issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar) as "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     cast("issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar) as "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     cast("description of issue - including any immediate resolution (include item values)" as varchar) as "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     cast("further resolution notes from ops, vendor team, am team, ops team (including details of charges)" as varchar) as "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at
2022-01-27 17:15:54 normalization > from __dbt__cte__october_2021_ab1
2022-01-27 17:15:54 normalization > -- october_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > ),  __dbt__cte__october_2021_ab3 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to build a hash column based on the values of this record
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__october_2021_ab2
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     md5(cast(coalesce(cast(date as varchar), '') || '-' || coalesce(cast(client as varchar), '') || '-' || coalesce(cast(vendor as varchar), '') || '-' || coalesce(cast(service as varchar), '') || '-' || coalesce(cast(location as varchar), '') || '-' || coalesce(cast(reporter as varchar), '') || '-' || coalesce(cast("user email" as varchar), '') || '-' || coalesce(cast("added to ia?" as varchar), '') || '-' || coalesce(cast("order number" as varchar), '') || '-' || coalesce(cast("contacted by?" as varchar), '') || '-' || coalesce(cast("ops team resolved" as varchar), '') || '-' || coalesce(cast("customer impacting?" as varchar), '') || '-' || coalesce(cast("mins if late/ early" as varchar), '') || '-' || coalesce(cast("added to tracker (ops)" as varchar), '') || '-' || coalesce(cast("am team acknowledge/ resolved" as varchar), '') || '-' || coalesce(cast("added to invoice tracker - (am)" as varchar), '') || '-' || coalesce(cast("added to invoice tracker (vendor)" as varchar), '') || '-' || coalesce(cast("vendor team acknowledge/ resolved" as varchar), '') || '-' || coalesce(cast("issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar), '') || '-' || coalesce(cast("issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar), '') || '-' || coalesce(cast("description of issue - including any immediate resolution (include item values)" as varchar), '') || '-' || coalesce(cast("further resolution notes from ops, vendor team, am team, ops team (including details of charges)" as varchar), '') as varchar)) as _airbyte_october_2021_hashid,
2022-01-27 17:15:54 normalization >     tmp.*
2022-01-27 17:15:54 normalization > from __dbt__cte__october_2021_ab2 tmp
2022-01-27 17:15:54 normalization > -- october_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > )-- Final base SQL model
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__october_2021_ab3
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     date,
2022-01-27 17:15:54 normalization >     client,
2022-01-27 17:15:54 normalization >     vendor,
2022-01-27 17:15:54 normalization >     service,
2022-01-27 17:15:54 normalization >     location,
2022-01-27 17:15:54 normalization >     reporter,
2022-01-27 17:15:54 normalization >     "user email",
2022-01-27 17:15:54 normalization >     "added to ia?",
2022-01-27 17:15:54 normalization >     "order number",
2022-01-27 17:15:54 normalization >     "contacted by?",
2022-01-27 17:15:54 normalization >     "ops team resolved",
2022-01-27 17:15:54 normalization >     "customer impacting?",
2022-01-27 17:15:54 normalization >     "mins if late/ early",
2022-01-27 17:15:54 normalization >     "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at,
2022-01-27 17:15:54 normalization >     _airbyte_october_2021_hashid
2022-01-27 17:15:54 normalization > from __dbt__cte__october_2021_ab3
2022-01-27 17:15:54 normalization > -- october_2021 from "feedr".customer_issue_tracker._airbyte_raw_october_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization >   );
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.559837 (Thread-16): SQL status: BEGIN in 0.24 seconds
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.560019 (Thread-16): Using redshift connection "model.airbyte_utils.may_2021".
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.560145 (Thread-16): On model.airbyte_utils.may_2021: /* {"app": "dbt", "dbt_version": "0.21.1", "profile_name": "normalize", "target_name": "prod", "node_id": "model.airbyte_utils.may_2021"} */
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization >   create  table
2022-01-27 17:15:54 normalization >     "feedr".customer_issue_tracker."may_2021__dbt_tmp"
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization >       compound sortkey(_airbyte_emitted_at)
2022-01-27 17:15:54 normalization >   as (
2022-01-27 17:15:54 normalization >     
2022-01-27 17:15:54 normalization > with __dbt__cte__may_2021_ab1 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to parse JSON blob stored in a single column and extract into separated field columns as described by the JSON Schema
2022-01-27 17:15:54 normalization > -- depends_on: "feedr".customer_issue_tracker._airbyte_raw_may_2021
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Client', true) != '' then json_extract_path_text(_airbyte_data, 'Client', true) end as client,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Vendor', true) != '' then json_extract_path_text(_airbyte_data, 'Vendor', true) end as vendor,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Service', true) != '' then json_extract_path_text(_airbyte_data, 'Service', true) end as service,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Reporter', true) != '' then json_extract_path_text(_airbyte_data, 'Reporter', true) end as reporter,
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'User email', true) != '' then json_extract_path_text(_airbyte_data, 'User email', true) end as "user email",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to IA?', true) != '' then json_extract_path_text(_airbyte_data, 'Added to IA?', true) end as "added to ia?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Complete by?', true) != '' then json_extract_path_text(_airbyte_data, 'Complete by?', true) end as "complete by?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Order number', true) != '' then json_extract_path_text(_airbyte_data, 'Order number', true) end as "order number",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Contacted by?', true) != '' then json_extract_path_text(_airbyte_data, 'Contacted by?', true) end as "contacted by?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Date of issue', true) != '' then json_extract_path_text(_airbyte_data, 'Date of issue', true) end as "date of issue",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Ops team resolved', true) != '' then json_extract_path_text(_airbyte_data, 'Ops team resolved', true) end as "ops team resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Customer impacting?', true) != '' then json_extract_path_text(_airbyte_data, 'Customer impacting?', true) end as "customer impacting?",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Mins if late/ early', true) != '' then json_extract_path_text(_airbyte_data, 'Mins if late/ early', true) end as "mins if late/ early",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to tracker (ops)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to tracker (ops)', true) end as "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'AM Team Acknowledge/ Resolved', true) != '' then json_extract_path_text(_airbyte_data, 'AM Team Acknowledge/ Resolved', true) end as "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to invoice tracker - (AM)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to invoice tracker - (AM)', true) end as "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Added to invoice tracker (Vendor)', true) != '' then json_extract_path_text(_airbyte_data, 'Added to invoice tracker (Vendor)', true) end as "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Vendor Team Acknowledge/ Resolved', true) != '' then json_extract_path_text(_airbyte_data, 'Vendor Team Acknowledge/ Resolved', true) end as "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Issue reason 2
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) != '' then json_extract_path_text(_airbyte_data, 'Issue reason 2
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) end as "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Issue reason 1 
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) != '' then json_extract_path_text(_airbyte_data, 'Issue reason 1 
2022-01-27 17:15:54 normalization > (Must match Internal admin)', true) end as "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Description of issue - including any immediate resolution (include item values)', true) != '' then json_extract_path_text(_airbyte_data, 'Description of issue - including any immediate resolution (include item values)', true) end as "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     case when json_extract_path_text(_airbyte_data, 'Further resolution notes from Ops, Vendor team, AM team, ops team (including details of charges)', true) != '' then json_extract_path_text(_airbyte_data, 'Further resolution notes from Ops, Vendor team, AM team, ops team (including details of charges)', true) end as "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at
2022-01-27 17:15:54 normalization > from "feedr".customer_issue_tracker._airbyte_raw_may_2021 as table_alias
2022-01-27 17:15:54 normalization > -- may_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > ),  __dbt__cte__may_2021_ab2 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to cast each column to its adequate SQL type converted from the JSON schema type
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__may_2021_ab1
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     cast(client as varchar) as client,
2022-01-27 17:15:54 normalization >     cast(vendor as varchar) as vendor,
2022-01-27 17:15:54 normalization >     cast(service as varchar) as service,
2022-01-27 17:15:54 normalization >     cast(reporter as varchar) as reporter,
2022-01-27 17:15:54 normalization >     cast("user email" as varchar) as "user email",
2022-01-27 17:15:54 normalization >     cast("added to ia?" as varchar) as "added to ia?",
2022-01-27 17:15:54 normalization >     cast("complete by?" as varchar) as "complete by?",
2022-01-27 17:15:54 normalization >     cast("order number" as varchar) as "order number",
2022-01-27 17:15:54 normalization >     cast("contacted by?" as varchar) as "contacted by?",
2022-01-27 17:15:54 normalization >     cast("date of issue" as varchar) as "date of issue",
2022-01-27 17:15:54 normalization >     cast("ops team resolved" as varchar) as "ops team resolved",
2022-01-27 17:15:54 normalization >     cast("customer impacting?" as varchar) as "customer impacting?",
2022-01-27 17:15:54 normalization >     cast("mins if late/ early" as varchar) as "mins if late/ early",
2022-01-27 17:15:54 normalization >     cast("added to tracker (ops)" as varchar) as "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     cast("am team acknowledge/ resolved" as varchar) as "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     cast("added to invoice tracker - (am)" as varchar) as "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     cast("added to invoice tracker (vendor)" as varchar) as "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     cast("vendor team acknowledge/ resolved" as varchar) as "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     cast("issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar) as "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     cast("issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar) as "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     cast("description of issue - including any immediate resolution (include item values)" as varchar) as "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     cast("further resolution notes from ops, vendor team, am team, ops team (including details of charges)" as varchar) as "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at
2022-01-27 17:15:54 normalization > from __dbt__cte__may_2021_ab1
2022-01-27 17:15:54 normalization > -- may_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > ),  __dbt__cte__may_2021_ab3 as (
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > -- SQL model to build a hash column based on the values of this record
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__may_2021_ab2
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     md5(cast(coalesce(cast(client as varchar), '') || '-' || coalesce(cast(vendor as varchar), '') || '-' || coalesce(cast(service as varchar), '') || '-' || coalesce(cast(reporter as varchar), '') || '-' || coalesce(cast("user email" as varchar), '') || '-' || coalesce(cast("added to ia?" as varchar), '') || '-' || coalesce(cast("complete by?" as varchar), '') || '-' || coalesce(cast("order number" as varchar), '') || '-' || coalesce(cast("contacted by?" as varchar), '') || '-' || coalesce(cast("date of issue" as varchar), '') || '-' || coalesce(cast("ops team resolved" as varchar), '') || '-' || coalesce(cast("customer impacting?" as varchar), '') || '-' || coalesce(cast("mins if late/ early" as varchar), '') || '-' || coalesce(cast("added to tracker (ops)" as varchar), '') || '-' || coalesce(cast("am team acknowledge/ resolved" as varchar), '') || '-' || coalesce(cast("added to invoice tracker - (am)" as varchar), '') || '-' || coalesce(cast("added to invoice tracker (vendor)" as varchar), '') || '-' || coalesce(cast("vendor team acknowledge/ resolved" as varchar), '') || '-' || coalesce(cast("issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar), '') || '-' || coalesce(cast("issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)" as varchar), '') || '-' || coalesce(cast("description of issue - including any immediate resolution (include item values)" as varchar), '') || '-' || coalesce(cast("further resolution notes from ops, vendor team, am team, ops team (including details of charges)" as varchar), '') as varchar)) as _airbyte_may_2021_hashid,
2022-01-27 17:15:54 normalization >     tmp.*
2022-01-27 17:15:54 normalization > from __dbt__cte__may_2021_ab2 tmp
2022-01-27 17:15:54 normalization > -- may_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization > )-- Final base SQL model
2022-01-27 17:15:54 normalization > -- depends_on: __dbt__cte__may_2021_ab3
2022-01-27 17:15:54 normalization > select
2022-01-27 17:15:54 normalization >     client,
2022-01-27 17:15:54 normalization >     vendor,
2022-01-27 17:15:54 normalization >     service,
2022-01-27 17:15:54 normalization >     reporter,
2022-01-27 17:15:54 normalization >     "user email",
2022-01-27 17:15:54 normalization >     "added to ia?",
2022-01-27 17:15:54 normalization >     "complete by?",
2022-01-27 17:15:54 normalization >     "order number",
2022-01-27 17:15:54 normalization >     "contacted by?",
2022-01-27 17:15:54 normalization >     "date of issue",
2022-01-27 17:15:54 normalization >     "ops team resolved",
2022-01-27 17:15:54 normalization >     "customer impacting?",
2022-01-27 17:15:54 normalization >     "mins if late/ early",
2022-01-27 17:15:54 normalization >     "added to tracker (ops)",
2022-01-27 17:15:54 normalization >     "am team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     "added to invoice tracker - (am)",
2022-01-27 17:15:54 normalization >     "added to invoice tracker (vendor)",
2022-01-27 17:15:54 normalization >     "vendor team acknowledge/ resolved",
2022-01-27 17:15:54 normalization >     "issue reason 2
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     "issue reason 1 
2022-01-27 17:15:54 normalization > (must match internal admin)",
2022-01-27 17:15:54 normalization >     "description of issue - including any immediate resolution (include item values)",
2022-01-27 17:15:54 normalization >     "further resolution notes from ops, vendor team, am team, ops team (including details of charges)",
2022-01-27 17:15:54 normalization >     _airbyte_ab_id,
2022-01-27 17:15:54 normalization >     _airbyte_emitted_at,
2022-01-27 17:15:54 normalization >     getdate() as _airbyte_normalized_at,
2022-01-27 17:15:54 normalization >     _airbyte_may_2021_hashid
2022-01-27 17:15:54 normalization > from __dbt__cte__may_2021_ab3
2022-01-27 17:15:54 normalization > -- may_2021 from "feedr".customer_issue_tracker._airbyte_raw_may_2021
2022-01-27 17:15:54 normalization > where 1 = 1
2022-01-27 17:15:54 normalization >   );
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.572777 (Thread-4): Postgres error: relation "customer_issue_tracker._airbyte_raw_july_2021" does not exist
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.572993 (Thread-4): On model.airbyte_utils.july_2021: ROLLBACK
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.602595 (Thread-4): finished collecting timing info
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.602889 (Thread-4): On model.airbyte_utils.july_2021: Close
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.603575 (Thread-4): Database Error in model july_2021 (models/generated/airbyte_tables/customer_issue_tracker/july_2021.sql)
2022-01-27 17:15:54 normalization >   relation "customer_issue_tracker._airbyte_raw_july_2021" does not exist
2022-01-27 17:15:54 normalization >   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/customer_issue_tracker/july_2021.sql
2022-01-27 17:15:54 normalization > Traceback (most recent call last):
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/adapters/postgres/connections.py", line 56, in exception_handler
2022-01-27 17:15:54 normalization >     yield
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/adapters/sql/connections.py", line 80, in add_query
2022-01-27 17:15:54 normalization >     cursor.execute(sql, bindings)
2022-01-27 17:15:54 normalization > psycopg2.errors.UndefinedTable: relation "customer_issue_tracker._airbyte_raw_july_2021" does not exist
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > The above exception was the direct cause of the following exception:
2022-01-27 17:15:54 normalization > 
2022-01-27 17:15:54 normalization > Traceback (most recent call last):
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/task/base.py", line 348, in safe_run
2022-01-27 17:15:54 normalization >     result = self.compile_and_execute(manifest, ctx)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/task/base.py", line 291, in compile_and_execute
2022-01-27 17:15:54 normalization >     result = self.run(ctx.node, manifest)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/task/base.py", line 393, in run
2022-01-27 17:15:54 normalization >     return self.execute(compiled_node, manifest)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/task/run.py", line 249, in execute
2022-01-27 17:15:54 normalization >     result = MacroGenerator(materialization_macro, context)()
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/clients/jinja.py", line 333, in __call__
2022-01-27 17:15:54 normalization >     return self.call_macro(*args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/clients/jinja.py", line 260, in call_macro
2022-01-27 17:15:54 normalization >     return macro(*args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/runtime.py", line 675, in __call__
2022-01-27 17:15:54 normalization >     return self._invoke(arguments, autoescape)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/runtime.py", line 679, in _invoke
2022-01-27 17:15:54 normalization >     rv = self._func(*arguments)
2022-01-27 17:15:54 normalization >   File "<template>", line 70, in macro
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/sandbox.py", line 462, in call
2022-01-27 17:15:54 normalization >     return __context.call(__obj, *args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/runtime.py", line 290, in call
2022-01-27 17:15:54 normalization >     return __obj(*args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/clients/jinja.py", line 333, in __call__
2022-01-27 17:15:54 normalization >     return self.call_macro(*args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/clients/jinja.py", line 260, in call_macro
2022-01-27 17:15:54 normalization >     return macro(*args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/runtime.py", line 675, in __call__
2022-01-27 17:15:54 normalization >     return self._invoke(arguments, autoescape)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/runtime.py", line 679, in _invoke
2022-01-27 17:15:54 normalization >     rv = self._func(*arguments)
2022-01-27 17:15:54 normalization >   File "<template>", line 41, in macro
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/sandbox.py", line 462, in call
2022-01-27 17:15:54 normalization >     return __context.call(__obj, *args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/jinja2/runtime.py", line 290, in call
2022-01-27 17:15:54 normalization >     return __obj(*args, **kwargs)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/adapters/base/impl.py", line 226, in execute
2022-01-27 17:15:54 normalization >     return self.connections.execute(
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/adapters/sql/connections.py", line 131, in execute
2022-01-27 17:15:54 normalization >     _, cursor = self.add_query(sql, auto_begin)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/adapters/sql/connections.py", line 87, in add_query
2022-01-27 17:15:54 normalization >     return connection, cursor
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/contextlib.py", line 131, in __exit__
2022-01-27 17:15:54 normalization >     self.gen.throw(type, value, traceback)
2022-01-27 17:15:54 normalization >   File "/usr/local/lib/python3.8/site-packages/dbt/adapters/postgres/connections.py", line 67, in exception_handler
2022-01-27 17:15:54 normalization >     raise dbt.exceptions.DatabaseException(str(e).strip()) from e
2022-01-27 17:15:54 normalization > dbt.exceptions.DatabaseException: Database Error in model july_2021 (models/generated/airbyte_tables/customer_issue_tracker/july_2021.sql)
2022-01-27 17:15:54 normalization >   relation "customer_issue_tracker._airbyte_raw_july_2021" does not exist
2022-01-27 17:15:54 normalization >   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/customer_issue_tracker/july_2021.sql
2022-01-27 17:15:54 normalization > 2022-01-27 17:15:41.605872 (Thread-4): 17:15:41 | 13 of 13 ERROR creating table model customer_issue_tracker.july_2021.

2022-01-27 17:15:54 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):177 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    ... 3 more
    Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
        at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:159) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:45) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]
2022-01-27 17:15:54 INFO i.a.w.t.TemporalAttemptExecution(get):144 - Stopping cancellation check scheduling...
2022-01-27 17:15:54 WARN i.t.i.s.POJOActivityTaskHandler(activityFailureToResult):363 - Activity failure. ActivityId=0901cef9-2da4-3378-b56e-b8fc9c05619e, activityType=Normalize, attempt=1
java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
    at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
    at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:142) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:97) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at jdk.internal.reflect.GeneratedMethodAccessor176.invoke(Unknown Source) ~[?:?]
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
    at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:286) ~[temporal-sdk-1.6.0.jar:?]
    at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:252) ~[temporal-sdk-1.6.0.jar:?]
    at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:209) ~[temporal-sdk-1.6.0.jar:?]
    at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:193) ~[temporal-sdk-1.6.0.jar:?]
    at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:151) ~[temporal-sdk-1.6.0.jar:?]
    at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:73) ~[temporal-sdk-1.6.0.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
    at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    ... 1 more
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
    ... 1 more
    Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
        at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:159) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:45) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:174) ~[io.airbyte-airbyte-workers-0.35.7-alpha.jar:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]

replace this with your long log output here



</details>

## Steps to Reproduce
1.
2.
3.

## Are you willing to submit a PR?
<!--- 
We accept contributions! 
Don't feel pressured, but if you want to contribute we can help you by giving some tips, 
highlighting the necessary code change or explaining any relevant point your feature will impact.
You can also send questions on #dev Slack channel.

We understand if you can't submit a PR and we're tremendously grateful
that you've already contributed by suggesting a new feature.
-->
Remove this with your answer.
alafanechere commented 2 years ago

Hi @nthbooth-feedr, do you have data in your google sheet for July 2021? Looks like the normalization misses _airbyte_raw_july_2021 table to work.

nthbooth-feedr commented 2 years ago

We did have data in that shet. There were a number of sheets that it failed on and they did all have data in.

marcosmarxm commented 2 years ago
2022-01-27 17:14:53 source > available_sheets: ['SHEET INSTRUCTIONS', 'January 2022', 'December 2021', 'October 2021', 'November 2021', 'June 2021', 'September 2021', 'July 2021 ', 'August 2021', 'On call incident report', 'Template', 'May 2021', 'April 2021', 'March 2021', 'March 2021 old', 'Feb 2021', 'January 2021', 'December 2020', 'November 2020', 'September', 'October', 'Validation', 'August', 'July', 'June', 'March', 'May', 'Feb', 'January', 'December', 'November', 'Archive: Mar - Oct ']
2022-01-27 17:14:53 source > Row counts: {'SHEET INSTRUCTIONS': 1000, 'January 2022': 2001, 'December 2021': 1011, 'October 2021': 2079, 'November 2021': 2098, 'June 2021': 882, 'September 2021': 2017, 'July 2021 ': 756, 'August 2021': 768, 'On call incident report': 2004, 'Template': 2004, 'May 2021': 919, 'April 2021': 986, 'March 2021': 996, 'March 2021 old': 2003, 'Feb 2021': 1998, 'January 2021': 2000, 'December 2020': 2000, 'November 2020': 2005, 'September': 2004, 'October': 2003, 'Validation': 1003, 'August': 2001, 'July': 1995, 'June': 1999, 'March': 1059, 'May': 989, 'Feb': 2139, 'January': 2144, 'December': 69, 'November': 138, 'Archive: Mar - Oct ': 2509}

The July 2021 has a additional space at the end of the sheet name maybe this is causing the issue.

marcosmarxm commented 1 year ago

Closed due inactivty. @nthbooth-feedr please open a new issue if the issue persists.