confluentinc / kafka-connect-jdbc

Kafka Connect connector for JDBC-compatible databases
Other
1.01k stars 953 forks source link

Support for PostgreSQL's OID data type #1378

Open developster opened 6 months ago

developster commented 6 months ago

At the moment this plugin does not support incremental querying for tables with an OID primary key.

The problem is that OID is defined at the moment as an unsigned 4 byte integer but the initial offset is a negative number.

The first query will look like this: SELECT * FROM "public"."config" WHERE "public"."config"."oid" > -1 ORDER BY "public"."config"."oid" ASC And PostgreSQL will respond with: OID out of range.

Way to overcome this problem:

Suppose the database has all tables with OID primary keys. It would be nice to be able to customize the condition in WHERE clause through a configuration parameter (condition_pattern?) the default being '%s > ?'. In that case it would be possible to do the casts for all the tables with cast(%s as int8) > ?.