Clickhouse Raw Table: Change order by to batch ID

PeerDB-io / peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

https://peerdb.io

Other

2.17k stars 88 forks source link

Clickhouse Raw Table: Change order by to batch ID #2102

Closed Amogh-Bharadwaj closed 6 days ago

Amogh-Bharadwaj commented 6 days ago

This PR changes the table creation command of the raw table for Clickhouse. Instead of ordering by _peerdb_uid, we now order by _peerdb_batch_id The goal is to improve the performance of SELECTs from the raw table during normalize

Amogh-Bharadwaj commented 6 days ago

following here now: https://github.com/PeerDB-io/peerdb/pull/2104