toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.16k stars 180 forks source link

Fix poll_db crash when notifications payload indeces is None #537

Closed mpaolino closed 3 months ago

mpaolino commented 6 months ago

Every time poll_db() tries to process a notification payload that has indices set to None it crashes. This can happen for several reasons, outdated triggers for instance, or due to other non-tracked changes in the DB pgsync listens to. For example a TRUNCATE in a table of the same DB that pgsync monitors but does not sync.

This PR fixes this by adding a simple check for a None value before trying to search the payload indices list.

Adding a test is challenging because the current code does not provide a way to control/exit the db_poll thread loop, hence you cannot exit the loop from a test once started. This requires a refactor that I think is out of the scope of this fix.

BTW, you should really think about having a stable branch with the latest release, main is currently broken and tests do not pass. This makes contributions much harder for other devs because there is no simple way to verify the fixes do not introduce secondary effects. You have to checkout the tag, cherry pick the fix commit, run the tests, and then make a PR against main.

Thanks for all your hard work and dedication to this project @toluaina, it's very useful and very much appreciated.

mpaolino commented 3 months ago

@toluaina is this project active?

toluaina commented 3 months ago

@toluaina is this project active?

Apologies, I've had very important life commitments. Yes it's is very much active.

mpaolino commented 3 months ago

Thank you again!