coderabbitai[bot]
commented
1 month ago Walkthrough
The changes in the pull request focus on two main areas: the addition of new indexes in the PostgreSQL migration script and enhancements to the Watch struct in the internal/storage/postgres/watch.go file. The migration introduces two new indexes to improve query performance on the attributes and relation_tuples tables. Concurrent creation and rollback mechanisms are included. The Watch struct modifications implement an exponential backoff strategy and allow batch processing of transaction IDs, improving the efficiency of database change handling.
Changes
| File Path | Change Summary |
|---|---|
| internal/storage/postgres/migrations/20241018182912_add_watch_indexes.sql | - Added two new indexes: idx_attributes_txid and idx_relation_tuples_txid for improved query performance.- Included rollback mechanism for concurrent dropping of indexes. |
| internal/storage/postgres/watch.go | - Added sleep variable of type *time.Timer for exponential backoff.- Updated Watch method with maxSleepDuration and sleepDuration.- Modified getRecentXIDs to include a limit parameter.- Replaced getChanges with getBatchChanges, accepting a slice of types.XID8 values.- Adjusted SQL queries to handle batch processing and added logging for transaction IDs. |
Poem
🐇 In the land of code, where changes bloom,
A timer now ticks, dispelling the gloom.
Batch processing hops, like rabbits in play,
Fetching IDs swiftly, come what may.
With exponential waits, we’ll dance through the night,
In the world of Postgres, our future is bright! 🌟
The transaction fetching logic is refactored to improve performance by implementing bulk queries for retrieving recent transactions and associated data changes. Previously transactions were fetched one by one or in smaller batches, which led to performance bottlenecks, especially when processing a large number of changes. This refactor optimizes the querying process and reduces the number of database calls.
Updates:
Summary by CodeRabbit
New Features
Bug Fixes