Cassandra Helm: debug.log not works and empty

[antikilahdjs]

Name and Version

bitnami/cassandra 4.1.0-debian-11-r6

What steps will reproduce the bug?

1 - Install Cassandra Helm Charts using normal way 2 - Enable diagnosticmode to TRUE or just enable the environment BITNAMI_DEBUG to TRUE 3 - Check debug.log in the path /opt/bitnami/cassandra/logs

Are you using any custom parameters or values?

          if [[ $POD_NAME =~ (.*)-0$ ]]; then
              echo "Setting node as password seeder"
              export CASSANDRA_PASSWORD_SEEDER=yes
          else
              # Only node 0 will execute the startup initdb scripts
              export CASSANDRA_IGNORE_INITDB_SCRIPTS=1
          fi

          /opt/bitnami/scripts/cassandra/entrypoint.sh
          /opt/bitnami/scripts/cassandra/run.sh
      ports:
        - name: intra
          containerPort: 7000
          protocol: TCP
        - name: tls
          containerPort: 7001
          protocol: TCP
        - name: jmx
          containerPort: 7199
          protocol: TCP
        - name: cql
          containerPort: 9042
          protocol: TCP
      env:
        - name: BITNAMI_DEBUG
          value: 'true'
        - name: CASSANDRA_CLUSTER_NAME
          value: cassandra-cluster
        - name: CASSANDRA_SEEDS
          value: >-
            cassandra-sts-master-0.cassandra-sts-master-headless.my-lab01-server.svc.cluster.local,cassandra-sts-master-1.cassandra-sts-master-headless.my-lab01-server.svc.cluster.local,cassandra-sts-master-2.cassandra-sts-master-headless.my-lab01-server.svc.cluster.local
        - name: CASSANDRA_PASSWORD
          valueFrom:
            secretKeyRef:
              name: cassandra-sts-master
              key: cassandra-password
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: CASSANDRA_USER
          value: cassandra
        - name: CASSANDRA_NUM_TOKENS
          value: '512'
        - name: CASSANDRA_DATACENTER
          value: datacenter1
        - name: CASSANDRA_ENDPOINT_SNITCH
          value: SimpleSnitch
        - name: CASSANDRA_KEYSTORE_LOCATION
          value: /opt/bitnami/cassandra/certs/keystore
        - name: CASSANDRA_TRUSTSTORE_LOCATION
          value: /opt/bitnami/cassandra/certs/truststore
        - name: CASSANDRA_RACK
          value: my-lab01-serverrack1
        - name: CASSANDRA_TRANSPORT_PORT_NUMBER
          value: '7000'
        - name: CASSANDRA_JMX_PORT_NUMBER
          value: '7199'
        - name: CASSANDRA_CQL_PORT_NUMBER
          value: '9042'
      resources: {}
      volumeMounts:
        - name: data
          mountPath: /bitnami/cassandra
        - name: init-db-cm
          mountPath: /docker-entrypoint-initdb.d/
        - name: configurations
          mountPath: /bitnami/cassandra/conf
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      imagePullPolicy: IfNotPresent
      securityContext:
        runAsUser: 1001
        runAsNonRoot: true
    - name: metrics
      image: 'docker.io/bitnami/cassandra-exporter:2.3.8-debian-11-r77'
      ports:
        - name: metrics
          containerPort: 8080
          protocol: TCP
        - name: jmx
          containerPort: 5555
          protocol: TCP
      resources: {}
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      imagePullPolicy: IfNotPresent

What is the expected behavior?

I am expecting the debug.log recorder a logs as well but after 2 days is not worked.

What do you see instead?

The debug.log needs to be filled to troubeshooting some errors or slow queries as well.

In my 3 instances the debug.log is empty, I am waited more than 2 days to make sure if it will be logged.

-rw-r--r-- 1 1001 root 166305 Jan 24 14:16 cassandra.log
-rw-r--r-- 1 1001 root      0 Jan 24 14:09 debug.log
-rw-r--r-- 1 1001 root  63408 Jan 24 14:20 gc.log.0.current

Additional information

cassandra 13:45:37.12 
 cassandra 13:45:50.39 Welcome to the Bitnami cassandra container
 cassandra 13:46:03.35 Subscribe to project updates by watching https://github.com/bitnami/containers
 cassandra 13:46:16.93 Submit issues and feature requests at https://github.com/bitnami/containers/issues
 cassandra 13:46:30.69 
 cassandra 13:46:43.97 INFO  ==> ** Starting Cassandra setup **
 cassandra 13:47:24.16 WARN  ==> CASSANDRA_HOST not set, defaulting to system hostname
 cassandra 13:48:03.81 INFO  ==> Validating settings in CASSANDRA_* env vars..
 cassandra 13:51:25.86 INFO  ==> Initializing Cassandra database...
 cassandra 13:52:33.58 DEBUG ==> No injected cassandra-env.sh file found. Creating default cassandra-env.sh file
 cassandra 13:53:00.05 DEBUG ==> No injected jvm8-server.options file found. Creating default jvm8-server.options file
 cassandra 13:53:26.41 DEBUG ==> No injected hotspot_compiler file found. Creating default hotspot_compiler file
 cassandra 13:53:52.40 DEBUG ==> No injected cassandra-topology.properties.example file found. Creating default cassandra-topology.properties.example file
 cassandra 13:54:18.88 DEBUG ==> Found cassandra.yaml. Skipping default
 cassandra 13:54:31.75 DEBUG ==> No injected cassandra-jaas.config file found. Creating default cassandra-jaas.config file
 cassandra 13:54:58.59 DEBUG ==> No injected metrics-reporter-config-sample.yaml file found. Creating default metrics-reporter-config-sample.yaml file
 cassandra 13:55:24.76 DEBUG ==> No injected triggers/README.txt file found. Creating default triggers/README.txt file
 cassandra 13:55:50.90 DEBUG ==> No injected cassandra-rackdc.properties file found. Creating default cassandra-rackdc.properties file
 cassandra 13:56:18.07 DEBUG ==> No injected jvm8-clients.options file found. Creating default jvm8-clients.options file
 cassandra 13:56:44.89 DEBUG ==> No injected logback-tools.xml file found. Creating default logback-tools.xml file
 cassandra 13:57:11.86 DEBUG ==> No injected README.txt file found. Creating default README.txt file
 cassandra 13:57:38.06 DEBUG ==> No injected jvm-clients.options file found. Creating default jvm-clients.options file
 cassandra 13:58:04.79 DEBUG ==> No injected logback.xml file found. Creating default logback.xml file
 cassandra 13:58:31.79 DEBUG ==> No injected jvm11-server.options file found. Creating default jvm11-server.options file
 cassandra 13:58:57.97 DEBUG ==> No injected credentials.sample file found. Creating default credentials.sample file
 cassandra 13:59:25.19 DEBUG ==> No injected commitlog_archiving.properties file found. Creating default commitlog_archiving.properties file
 cassandra 13:59:51.22 DEBUG ==> No injected cqlshrc.sample file found. Creating default cqlshrc.sample file
 cassandra 14:00:18.72 DEBUG ==> No injected jvm11-clients.options file found. Creating default jvm11-clients.options file
 cassandra 14:00:44.70 DEBUG ==> No injected jvm-server.options file found. Creating default jvm-server.options file
 cassandra 14:01:11.32 DEBUG ==> cassandra.yaml mounted. Skipping authentication method configuration
 cassandra 14:02:31.12 DEBUG ==> cassandra.yaml mounted. Skipping native and storage ports configuration
 cassandra 14:03:25.72 DEBUG ==> cassandra.yaml mounted. Skipping data directory configuration
 cassandra 14:03:38.91 DEBUG ==> cassandra.yaml mounted. Skipping cluster configuration
 cassandra 14:04:44.46 DEBUG ==> Ensuring expected directories/files exist...
 cassandra 14:06:02.17 INFO  ==> Deploying Cassandra with persisted data
 cassandra 14:06:28.07 INFO  ==> ** Cassandra setup finished! **
 cassandra 14:07:21.01 INFO  ==> ** Starting Cassandra **

[github-actions[bot]]

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

[CeliaGMqrz]

Hi @antikilahdjs, sorry for the delay.

Thanks for your report. I have been able to reproduce your issue. In fact, this option for the debug.log is disabled. Specifically this line of the Cassandra configuration

    <!-- <appender-ref ref="ASYNCDEBUGLOG" /> --> <!-- Comment this line to disable debug.log -->

It is NOT recommended to activate this option because log files accumulate on the container disk. I have created an internal task to get these logs printed into stdout/stderr.

Occasionally and alternatively, you can provide your config.xml using the extraVolumes parameter. You can create a config map from the configuration file and install the chart with these values.

kubectl create configmap cassandra-cm --from-file=./logback.xml 
kubectl get configmap cassandra-cm -o yaml

## @param extraVolumes Optionally specify extra list of additional volumes for cassandra container
##
extraVolumes: 
  - name: cassandra-config
    configMap:
       name: cassandra-cm
## @param extraVolumeMounts Optionally specify extra list of additional volumeMounts for cassandra container
##
extraVolumeMounts: 
  - name: cassandra-config
    mountPath: /bitnami/cassandra/conf

We'll leave this issue on-hold until our dev team works on the related task.

[antikilahdjs]

@CeliaGMqrz I hope you doing good.

I really appreciate that and your return about it. I will try to enable it as you sent.

[carrodher]

Hi @antikilahdjs, please note that setting the BITNAMI_DEBUG environment variable to true increase the verbosity in stdout from logs coming from the Bitnami initialization. This produces the DEBUG ==> entries you see in the console output, with the default value of the BITNAMI_DEBUG environment variable (false), the level of logs is higher so only the INFO ==> or WARN ==> messages will appear.

Being said that, what you mention about the debug.log file is not related to the Bitnami initialization logs but to the Cassandra logs themselves. As @CeliaGMqrz rightly said, in order to not increase the disk usage and therefore the size of the volumes, this log was disabled out of the box. We just created an internal task to provide an option to enable this option via a parameter, so users interested in those logs can enable it at their own risk knowing the consequences in terms of the size of the disk they may have.

[antikilahdjs]

@carrodher thank you so much and I got it . The debug.log is just for the critical troubleshoot and I needed it to check some slow queries in my internal environment but the debug.log is empty as well, so , if in future we can able to enable it will be great.

Thank you so much and thank @CeliaGMqrz , keep safe.

[ztolley]

Seems to have started in r22