PeerDB-io / peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
https://peerdb.io
Other
2.17k stars 88 forks source link

Numeric type: improved interface, fix Clickhouse support #2084

Closed Amogh-Bharadwaj closed 1 week ago

Amogh-Bharadwaj commented 1 week ago

This PR:

Previous behaviour for Clickhouse: Since 78 is higher than the maximum supported precision on Clickhouse, we would set it to PeerDB's chosen default precision (76) and default scale (38)

New behaviour: We first see that the precision is above max so we just change precision to the max of 76. Then we check if the new (precision, scale) tuple is valid. If not, then fallback to the behaviour from before and set it to defaults chosen by PeerDB. In this case it will now be valid, so the numeric on target will be Decimal(76, 70). Notice how we do not change the scale unlike above

Motivation: This allows us to now instead of setting a numeric like (78,0) to (76,36), to now just set it to (76,0) - which is more supportive than (76,36) when the incoming numerics are not going to have any fractional parts