search

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.1k stars 172 forks source link

Issue with Data Synchronization in pgsync #545

Open HalfSea opened 2 weeks ago

HalfSea commented 2 weeks ago

PGSync version: 3.1.0

Postgres version: 14.12

Elasticsearch/OpenSearch version: 8.13.4

Redis version: 6.0.16

Python version: 3.10.12

Problem Description: I encountered an issue while using pgsync (version 3.1.0) with PostgreSQL (version 14.12) and ElasticSearch (version 8.13.4). During the operation, ElasticSearch crashed, which also caused pgsync to stop. During this downtime, some data in a table in the PostgreSQL database was physically deleted. After restarting ElasticSearch and pgsync, new and modified data in the PostgreSQL database was synchronized correctly, but the physically deleted data was not removed in ElasticSearch, leading to data inconsistencies.

Error Message (if any):

HalfSea commented 2 weeks ago

I used a simple data table with only id and name fields to reproduce the issue but did not encounter the same problem. I will investigate further to find a reliable way to reproduce the issue and provide a detailed description.