airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.19k stars 4.14k forks source link

[destination-firestore] timestamp on the source converted to string on Firestore destination #32342

Open joemusashi opened 1 year ago

joemusashi commented 1 year ago

Connector Name

destination-firestore

Connector Version

0.1.1

What step the error happened?

During the sync

Relevant information

Description

Syncing data from a REST API to Firestore including a timestamp field with timezone with the right schema but not recognized by Google Firestore as timestamp but a string.

To Reproduce

Data from the REST API

[ { "name": "My product name", "price": 6.5, "updated_at": "2023-11-28T17:45:00Z" } ]

Detected schema

{ "$schema": "http://json-schema.org/schema#", "properties": { "name": { "type": "string" }, "price": { "type": "number" }, "updated_at": { "type": "string" } } }

Declared schema

{ "$schema": "http://json-schema.org/schema#", "properties": { "name": { "type": "string" }, "price": { "type": "number" }, "updated_at": { "airbyte_type": "timestamp_with_timezone", "format": "date-time", "type": "string" } } }

Expected behavior

The timestamp field from the source should be recognized as a timestamp on the Google Firestore side.

Thanks for your help!

Relevant log output

No response

Contribute

marcosmarxm commented 6 months ago

Which version of the Airbyte CDK are you using for your custom connector? Also, the Firestore community connector has a basic implementation. After reviewing the code, it appears it doesn't handle data types properly.

octavia-squidington-iii commented 5 days ago

At Airbyte, we seek to be clear about the project priorities and roadmap. This issue has not had any activity for 180 days, suggesting that it's not as critical as others. It's possible it has already been fixed. It is being marked as stale and will be closed in 20 days if there is no activity. To keep it open, please comment to let us know why it is important to you and if it is still reproducible on recent versions of Airbyte.