Write To Database Table Instead of Text File

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync

https://pgsync.com

MIT License

1.1k stars 172 forks source link

Write To Database Table Instead of Text File #529

Open jvanderen1 opened 4 months ago

jvanderen1 commented 4 months ago

Maintaining the generated pgsync file in the cloud has proven slightly cumbersome. I was wondering if we could have an option instead write the checkpoint into a database table? A table with a schema like:

Checkpoint Name	Value
`<user>_<database>_<index>`	`<Generated Hash>`

toluaina commented 3 weeks ago

How about moving this to Redis? I'd prefer to pollute the database as little as possible. I think I had a POC branch to move this to Redis

jvanderen1 commented 3 weeks ago

@toluaina That is fair. I ended up actually Dockerizing my application and maintaining this checkpoint file within a persistent cache layer.