https://luminousmen.com/post/change-data-capture?utterances=86150fb214256cb379e189a2hSnt8JqqJorjuDYEDIfoDzkAtiVYXkP6KgdQ%2B%2BqVnEmxRh7iQ4todsCHUcjTGs6dH0Bw4qYK%2BGoPVA3E4uYIul9IvpYWY0G8D%2BgmgeClcZQi4QpXPE4McWBW%2Bkw%3D

[utterances-bot]

Change Data Capture (CDC) - Blog | luminousmen

How to detect, capture, and propagate changes from source databases to target systems in a real-time, modern design approach to CDC

https://luminousmen.com/post/change-data-capture?utterances=86150fb214256cb379e189a2hSnt8JqqJorjuDYEDIfoDzkAtiVYXkP6KgdQ%2B%2BqVnEmxRh7iQ4todsCHUcjTGs6dH0Bw4qYK%2BGoPVA3E4uYIul9IvpYWY0G8D%2BgmgeClcZQi4QpXPE4McWBW%2Bkw%3D

[simkimsia]

Thanks for writing this 🙏

2 Questions:

Question 1 how to do the simplest version ofCDC as you described above if using a typical RDBMS web framework like Django or Rails? Assume I’m using Postgres and Django since your about says you use python?

Question 2 suppose same context in previous question instead of like you said tracking delta changes from database changes but this time track changes via event driven? Is that sound?

Thanks for writing

[luminousmen]

Hey @simkimsia, thank you for kind words!

1. It doesn't really matter which web framework you're using. It all depends on RDBMS, for MySQL you can use binlog connector, I'm not sure PostgreSQL has something like that - I recall it does not have replication built-in - but you probably can easily google that.
2. Didn't get the question. Are we talking about event sourcing?

[simkimsia]

@luminousmen for q2 yes, event sourcing.

[luminousmen]

@simkimsia, I don't have much experience with event sourcing, but the idea there is a little bit different afaik. CDC will record physical RDBMS changes like ChangeLog("order num": 1, "updated field": "order", "old value": "new", "new value": "ready"}, while event sourcing will give you domain level changes like Event("order num": 1, "event": "order completed"). Events are something sharable between microservices, as they exist at a far higher level of abstraction than data changes. What I'm trying to say is, event are meaningful to the domain, but data representing the entity can change without impacting the business meaning.

I guess you can in some cases use one instead of the other because their goals are very similar, but overall sometimes you will get some serious design problems if you choose not the right tool for the job. Hope it all makes sense :)