search

zendesk / maxwell

Maxwell's daemon, a mysql-to-json kafka producer
https://maxwells-daemon.io/
Other
4.05k stars 1.01k forks source link

Null column include filter doesn't seem to work, excludes all values #2110

Open sbracegirdle opened 2 months ago

sbracegirdle commented 2 months ago

Hi there, I have a filter rule that is effectively this:

exclude: *.*, include: mydb.User, include: mydb.Entity.someColumn=null

The User changes come through perfectly, but the entity data does not, regardless of someColumn value. I also tried alternative patterns like "null" and /^null$/ to no avail.

But reading the source code for filtering, it does look like there's some logic to handle null values when the pattern is null (https://github.com/zendesk/maxwell/blob/master/src/main/java/com/zendesk/maxwell/filtering/FilterColumnPattern.java)? Have I misinterpreted how it works, or is there an issue? I'm using the zendesk/maxwell:latest container.

If I can get it working I'd be happy to PR a docs change to add examples for null filtering.

osheroff commented 2 months ago

I tried the most obvious case (exclude: *.*, include: test.foo.a=null) and it seemed to work properly... can you put together a script or a test that repros the issue?