search

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.19k stars 181 forks source link

Issue with Data Synchronization in pgsync #545

Open HalfSea opened 5 months ago

HalfSea commented 5 months ago

PGSync version: 3.1.0

Postgres version: 14.12

Elasticsearch/OpenSearch version: 8.13.4

Redis version: 6.0.16

Python version: 3.10.12

Problem Description: I encountered an issue while using pgsync (version 3.1.0) with PostgreSQL (version 14.12) and ElasticSearch (version 8.13.4). During the operation, ElasticSearch crashed, which also caused pgsync to stop. During this downtime, some data in a table in the PostgreSQL database was physically deleted. After restarting ElasticSearch and pgsync, new and modified data in the PostgreSQL database was synchronized correctly, but the physically deleted data was not removed in ElasticSearch, leading to data inconsistencies.

Error Message (if any):

HalfSea commented 5 months ago

I used a simple data table with only id and name fields to reproduce the issue but did not encounter the same problem. I will investigate further to find a reliable way to reproduce the issue and provide a detailed description.