search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.96k stars 4.1k forks source link

[source-shopify] field type for processed_at in the orders stream is incorrect, should be date-time (timestamp) instead of string #37402

Open lennyxc opened 6 months ago

lennyxc commented 6 months ago

Connector Name

source-shopify

Connector Version

2.0.5

What step the error happened?

Other

Relevant information

The orders.json schema has processed_at set to the following:

"processed_at": {
      "type": ["null", "string"]
    }

I believe it should be the same format as created_at

"processed_at": {
      "type": ["null", "string"],
      "format": "date-time"
    }

As you can see from Shopify docs, it should only ever be DateTime: https://shopify.dev/docs/api/admin-graphql/2023-07/objects/Order#field-order-processedat

This issue is causing problems downstream, as I load into a BigQuery destination, the column is typed as a STRING instead of DATETIME.

Relevant log output

No response

Contribute

marcosmarxm commented 5 months ago

@lennyxc do you want to submit the change?