long2ice / meilisync

Realtime sync data from MySQL/PostgreSQL/MongoDB to Meilisearch
https://github.com/long2ice/meilisync
Apache License 2.0
285 stars 43 forks source link

Postgres source ignores changes #100

Closed MattExact closed 6 months ago

MattExact commented 6 months ago

The following lines are used to determine whether to add a change to the queue. Instead of return when the table is not in self.tables, it should be continue. Otherwise if the first change's table is not in the list of tables, it will ignore all other changes.

https://github.com/long2ice/meilisync/blob/e1444185f52712969a04903e0195dbec9c2559df/meilisync/source/postgres.py#L94-L98

For example if self.tables = ["bar"], then the following replication message would be ignored:

changes = [
    {
        "kind": "update",
        "schema": "public",
        "table": "foo",
        "columnnames": ["id", "attributes"],
        "columntypes": ["integer", "hstore"],
        "columnvalues": [123456, '"foo"=>"bar"'],
        "oldkeys": {"keynames": ["id"], "keytypes": ["integer"], "keyvalues": [123456]},
    },
    {
        "kind": "update",
        "schema": "public",
        "table": "bar",
        "columnnames": ["id", "content"],
        "columntypes": ["integer", "text"],
        "columnvalues": [456789, "Lorem ipsum"],
        "oldkeys": {"keynames": ["id"], "keytypes": ["integer"], "keyvalues": [456789]},
    },
]
long2ice commented 6 months ago

Thanks for report! Could you please make a PR?