search

bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
9k stars 9.21k forks source link

how to change zookeeper's logback.xml settings #18436

Open wltinan opened 1 year ago

wltinan commented 1 year ago

Name and Version

bitnami/zookeeper:3.8.1

What architecture are you using?

None

What steps will reproduce the bug?

helm install zookeeper bitnami/zookeeper

Are you using any custom parameters or values?

no, I don't

What is the expected behavior?

I'd like to see the log file(zookeeper.log) under /opt/bitnami/zookeeper/logs directory but no files are found. It seems container default output is standard output.

I saw a post about changing the log4j config (ZOO_LOG4J_PROP = 'INFO, ROLLINGFILE') but in zookeeper version 3.8.1, logback was used not log4j, so it was not suitable. How can I leave log files under /opt/bitnami/zookeeper/logs?

What do you see instead?

/opt/bitnami/zookeeper/conf/logback.xml

<?xml version="1.0"?>
<!--
 Copyright 2022 The Apache Software Foundation

 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.

 Define some default values that can be overridden by system properties
-->
<configuration>
  <!-- Uncomment this if you would like to expose Logback JMX beans -->
  <!--jmxConfigurator /-->
  <property name="zookeeper.console.threshold" value="ERROR"/>
  <property name="zookeeper.log.dir" value="."/>
  <property name="zookeeper.log.file" value="zookeeper.log"/>
  <property name="zookeeper.log.threshold" value="INFO"/>
  <property name="zookeeper.log.maxfilesize" value="256MB"/>
  <property name="zookeeper.log.maxbackupindex" value="20"/>
  <!--
    console
    Add "console" to root logger if you want to use this
  -->
  <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n</pattern>
    </encoder>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>${zookeeper.console.threshold}</level>
    </filter>
  </appender>
  <!--
    Add ROLLINGFILE to root logger to get log file output
  -->
  <!--appender name="ROLLINGFILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>${zookeeper.log.dir}/${zookeeper.log.file}</File>
    <encoder>
      <pattern>%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n</pattern>
    </encoder>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>${zookeeper.log.threshold}</level>
    </filter>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <maxIndex>${zookeeper.log.maxbackupindex}</maxIndex>
      <FileNamePattern>${zookeeper.log.dir}/${zookeeper.log.file}.%i</FileNamePattern>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <MaxFileSize>${zookeeper.log.maxfilesize}</MaxFileSize>
    </triggeringPolicy>
  </appender-->
  <!--
    Add TRACEFILE to root logger to get log file output
    Log TRACE level and above messages to a log file
  -->
  <!--property name="zookeeper.tracelog.dir" value="${zookeeper.log.dir}" />
  <property name="zookeeper.tracelog.file" value="zookeeper_trace.log" />
  <appender name="TRACEFILE" class="ch.qos.logback.core.FileAppender">
    <File>${zookeeper.tracelog.dir}/${zookeeper.tracelog.file}</File>
    <encoder>
      <pattern>%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n</pattern>
    </encoder>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>TRACE</level>
    </filter>
  </appender-->
  <!--
    zk audit logging
  -->
  <!--property name="zookeeper.auditlog.file" value="zookeeper_audit.log" />
  <property name="zookeeper.auditlog.threshold" value="INFO" />
  <property name="audit.logger" value="INFO, RFAAUDIT" />

  <appender name="RFAAUDIT" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>${zookeeper.log.dir}/${zookeeper.auditlog.file}</File>
    <encoder>
      <pattern>%d{ISO8601} %p %c{2}: %m%n</pattern>
    </encoder>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>${zookeeper.auditlog.threshold}</level>
    </filter>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <maxIndex>10</maxIndex>
      <FileNamePattern>${zookeeper.log.dir}/${zookeeper.auditlog.file}.%i</FileNamePattern>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <MaxFileSize>10MB</MaxFileSize>
    </triggeringPolicy>
  </appender>

  <logger name="org.apache.zookeeper.audit.Slf4jAuditLogger" additivity="false" level="${audit.logger}">
    <appender-ref ref="RFAAUDIT" />
  </logger-->
  <root level="INFO">
    <appender-ref ref="CONSOLE"/>
  </root>
</configuration>

I have no name!@kafka-log-kafka-zookeeper-0:/opt/bitnami/zookeeper/logs$ ls -al total 0 drwxrwxr-x 2 root root 6 Jun 18 15:36 . drwxr-xr-x 1 root root 18 Jun 18 15:36 ..

Additional information

No response

aoterolorenzo commented 1 year ago

Hi @wltinan,

Sorry for the delay, I wanted to perform some tests and ask the team about the issue. Unfortunately, I cannot see other way to do this for logback than adding logic to the chart/container in order to do that, or mounting a custom logback.xml file into the config directory directly.

Anyway, I consider that this should be supported by our container/chart logic a easy way to interact with the logging configuration file, so I will create a task for the team to implement this. We will reach you back here our workload allow us to address the task. Meanwhile, I will leave this issue marked as on-hold to prevent the stale bot to close it.

janhoy commented 1 year ago

Hi. Hope all is well. I don't know how far ahead you are with solving this issue, but adding my comment here since I'm working on achieving JSON format structured logging from Zookeeper PODs.

I have filed an issue to the Zookeeper project (https://issues.apache.org/jira/browse/ZOOKEEPER-4757) to add the needed jar dependency and logback appender snippet, plus a way to change appender-ref by system property. But a way to provide a custom logback.xml in a CM is probably necessary for many log customization requirements, such as enabling audit logging.

~Unfortunately the response from zookeeper project is mixed so far, so I also filed a similar issue to the zookeeper Dockerfile maintainer in https://github.com/31z4/zookeeper-docker/issues/154 to get some feedback.~ UPDATE The Zookeeper project turned down the PR.

As I just realized that bitnami maintains their own Zookeeper Dockerfile, I filed an issue in that project for JSON logging feature: https://github.com/bitnami/containers/issues/51921.

This helm chart should of course allow users to completely replace logback.xml with their own custom one, but ideally they should then use some template in order to preserve controlling log level and json logging with env.vars.