How to reduce Kafka logging for phoebus client, logback, slf4j

[tanviash]

@kasemir Hi Kay, Is there a way to reduce logs that show up when Phoebus client is started? Right now, when I start up the Phoebus client using 'java -jar', I see huge number of kafka-related logs. On the service side, I can reduce this logs with .properties file but I can't find a way to reduce logs on the client side.

[berryma4]

Just modify the logging level of the originating log entries. Or The whole namespace org.apache.kafka=WARN

On Fri, Apr 5, 2019, 15:57 tanviash notifications@github.com wrote:

@kasemir https://github.com/kasemir Hi Kay, Is there a way to reduce logs that show up when Phoebus client is started? Right now, when I start up the Phoebus client using 'java -jar', I see huge number of kafka-related logs. On the service side, I can reduce this logs with .properties file but I can't find a way to reduce logs on the client side.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/shroffk/phoebus/issues/627, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0IBclwXXjvx6XFd2EEuRSMEaiO5fxsks5vd6qjgaJpZM4cfn73 .

[kasemir]

Right. The default log settings are in https://github.com/shroffk/phoebus/blob/master/phoebus-product/src/main/resources/logging.properties

My last commit was actually related to reducing the Kafka log levels: https://github.com/shroffk/phoebus/commit/76bd041caeb41fcb7660d0fc43fe7b79d7792042

Now that's the default log config for the UI product. The alarm server has its own file, https://github.com/shroffk/phoebus/blob/master/services/alarm-server/src/main/resources/alarm_server_logging.properties

All Phoebus products, when started from the command line, should understand the option -logging option, see https://github.com/shroffk/phoebus/blob/master/phoebus-product/src/main/java/org/phoebus/product/Launcher.java#L163 or https://github.com/shroffk/phoebus/blob/master/services/alarm-logger/src/main/java/org/phoebus/alarm/logging/AlarmLoggingService.java#L34 or https://github.com/shroffk/phoebus/blob/master/services/alarm-server/src/main/java/org/phoebus/applications/alarm/server/AlarmServerMain.java#L494 So that way you can use your own log settings when starting tools via the command line. When starting tools via springboot, I'm sure there's some other way, but I don't know it.

[tanviash]

Thanks Kay & Eric.

[shroffk]

Hello, Since the Kafka libraries use log4j, adding the above changes in the logging.properties does not work.

17:16:43.507 [AlarmClientModel] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - [Consumer clientId=consumer-1, groupId=Alarm-ddb1528b-ecc1-4277-871a-7b081aba010b] READ_UNCOMMITTED fetch request for partition XF23ID_OPR-0 at offset 2732 to node xf23id1-srv3.cs.nsls2.local:9092 (id: 0 rack: null)

We could include an additional log4j.properties in conjunction with the logging.properties.

I was also looking into the prossibility of trying to use the same logging configuration file.

[kasemir]

This is one way to configure the Kafka logging since it's resistent to the basic java.util.logging:

1) Create a file logback.xml with for example this content:

<!-- Disable logback logging
       Used by kafka client lib
  -->
<configuration>
  <root level="off">
  </root>
</configuration>

2) Start phoebus with -Dlogback.configurationFile=/path/to/that/logback.xml

I'm lost in the detailed relationship between log4j and logback; configuring logback seemed the easiest way to suppress the numerous Kafka client lib messages.

[kasemir]

Just got another question about it, leaving open to make it easier to find.

[shroffk]

I think we need a tag, there are other issues which also might benefit from being marked so that they can be used as a reference.

[kasemir]

Like 'FAQ'? A 'label' to mark some tickets as issues that keep coming up, there's info/details/workarounds, but no simple code fix at this time?