search

scylladb / scylla-cdc-java

Apache License 2.0
24 stars 15 forks source link

Route Flogger logs to log4j in Connector #35

Closed avelanarius closed 3 years ago

avelanarius commented 3 years ago

Add routing of Flogger logs in Scylla CDC Source Connector to log4j.

Flogger is used in the underlying scylla-cdc-java library and before this change those logs were not forwarded to log4j. Debezium (as well as Kafka and Kafka Connect) use the log4j library.

To perform this routing, a dependency on flogger-log4j-backend is added. In static block flogger.backend_factory property is changed to use the log4j backend. This static block is placed before any other static initializations and it is present in two classes that can be loaded by Kafka Connect - ScyllaConnector and ScyllaConnectorTask. Such placement is very important, as it MUST be executed before any FluentLogger.forEnclosingClass() invocations (which read the System property).

I have done the following testing:

  1. Run the connector on Confluent Platform. Logs from underlying library were correctly forwarded.
  2. Run the connector on open-source Kafka (2.6.0), in Kafka Connect distributed mode (2 nodes). The first node loaded ScyllaConnector class (and afterwards the ScyllaConnectorTask class) and correctly forwarded the logs. The second node loaded only ScyllaConnectorTask class and also correctly forwarded the logs.
avelanarius commented 3 years ago

Merged in the new repo: https://github.com/scylladb/scylla-cdc-source-connector