search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.4k stars 3.97k forks source link

Source Bigquery GCS: Avro int64 are formatted as int #10240

Closed aymen-k closed 4 weeks ago

aymen-k commented 2 years ago
## Environment - **Airbyte version**: 0.35.5-alpha - **OS Version / Instance**: macOS - **Deployment**: Docker - **Source Connector and version**: - **Destination Connector and version**: BigQuery (denormalized typed struct) v0.2.7 - **Severity**: Low - **Step where error happened**: Sync job ## Current Behavior *Tell us what happens.* I use the GCS option to push data to BigQuery. In my source streams, I have an int64 for example -2671251624134333867 This field is formatted in the avro file and the value pushed to BigQuery is different. I got -37115307 ## Expected Behavior *Tell us what should happen.* I would expect the stream value to be the same as the Big Query one ## Logs *If applicable, please upload the logs from the failing operation. For sync jobs, you can download the full logs from the UI by going to the sync attempt page and clicking the download logs button at the top right of the logs display window.*
LOG ``` replace this with your long log output here ```
## Steps to Reproduce 1. 2. 3. ## Are you willing to submit a PR?

Yes, I will submit a PR

etsybaev commented 1 year ago

At first glance, it seems to be the same as this issue https://github.com/airbytehq/airbyte/issues/17564