search

singer-io / tap-chargebee

Singer.io tap for extracting data from the Chargebee API
https://www.singer.io/
GNU Affero General Public License v3.0
7 stars 32 forks source link

Found a difference between data arriving in Snowflake/Bigquery vs Postgres #34

Open FlorianDeliverect opened 3 years ago

FlorianDeliverect commented 3 years ago

Hi,

I was migrating to Snowflake from a Postgres-based analytics set-up and found a difference. The Postgres one is correct after checking in chargebee. I thought this might be a Snowflake error so I created another stitch account and sent my data to Bigquery. However there I have exactly the same data as in Snowflake.

I'm talking about the customer-table. I don't know how often it happens, but here is an example record for which I made a field by field comparison : https://docs.google.com/spreadsheets/d/19pjWtwEjfOY8s0gJdVEa1gBwopclo8CI68xJpURX_xM/edit#gid=0

Strangely for Snowflake/Bigquery the updated_at field equals the created_at field while that customer has received many changes since it's creation date. All fields seem to correspond with the state they had at creation time while in postgres the updated_at field is this month and the other fields match what I find in Chargebee. Another strange thing is that the field DELETED is set to TRUE in the Snowflake/Bigquery version, while I cannot find any customer_deleted event in the customer's event history.

If you have any idea what's wrong, let me know.

Florian

cb-nandita commented 3 years ago

@FlorianDeliverect , Hope you are doing well. Could you please check and confirm if there are any errors in the Stitch logs? Are all the syncs successful ?