search

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.1k stars 172 forks source link

Support for specifying custom keys to use for document _id #498

Open 4d11 opened 8 months ago

4d11 commented 8 months ago

PGSync version: 2.5.0

Postgres version: 15

Elasticsearch version: 7.8.0

Redis version: ..

Python version: 3.9.6

Problem Description: Is it feasible to add support for using custom _ids instead of using the table's specified ones. I have a use case where the table's primary keys are (item_id, item_version). I would like to only have the latest version of the record indexed in Elasticsearch while keeping older versions as a record in postgres. Specifying the _id as (item_id) would achieve this behaviour.

I will happy to pick up this task if there is no existing method

Error Message (if any):

vanqard commented 6 months ago

Did you solve this issue? I too need the ability to generate a custom _id but don't want to duplicate your work if you've already done it.

toluaina commented 3 weeks ago

Sorry, I'm late to respond to this. Custom primary keys are not supported. However, pgsync already handles composite primary keys so you don't have to do this. The _id is a concatenation of all primary keys on the table