search

zendesk / maxwell

Maxwell's daemon, a mysql-to-json kafka producer
https://maxwells-daemon.io/
Other
4.01k stars 1.01k forks source link

Schema capturing in maxwell #2055

Closed Shukla-Ankur closed 11 months ago

Shukla-Ankur commented 11 months ago

This thread is to understand how maxwell builds schemas. Fom the documentation, I understand that 3 tables capture the "base" schema and then newer changes are captured in schemas table. Assume I start maxwell on MySQL server for 1st time, and I am using init_position to start playing binlogs from an old time T1. There are DDLs after time T1 also.

  1. In this case, what is the "base" schema captured in database,table and column tables captured? Is it the current schema of tables in the databases?
  2. If base schema captured is current schema, then how does maxwell manage to process events from an older point in time where schema would be different for some tables?
osheroff commented 11 months ago

Hi Shukla, I'm gonna move this thread to discussions.