search

confluentinc / ksql

The database purpose-built for stream processing applications.
https://ksqldb.io
Other
66 stars 1.04k forks source link

/bin/ksql command return log4j error "log4j:ERROR Could not read configuration file" #3882

Open yodo-t opened 4 years ago

yodo-t commented 4 years ago

Describe the bug When I run /bin/ksql, following error occurs.

$ /bin/ksql http://localhost:8088

  log4j:ERROR Could not read configuration file from URL [file://etc/ksql/log4j-file.properties].
  java.net.UnknownHostException: etc
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
    at java.net.Socket.connect(Socket.java:589)
    at sun.net.ftp.impl.FtpClient.doConnect(FtpClient.java:962)
    at sun.net.ftp.impl.FtpClient.tryConnect(FtpClient.java:924)
    at sun.net.ftp.impl.FtpClient.connect(FtpClient.java:1019)
    at sun.net.ftp.impl.FtpClient.connect(FtpClient.java:1005)
    at sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:311)
    at sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:417)
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:557)
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
    at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
    at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
    at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
    at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
    at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
    at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
    at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
    at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
    at io.confluent.ksql.Ksql.<clinit>(Ksql.java:36)
  log4j:ERROR Ignoring configuration file [file://etc/ksql/log4j-file.properties].
  log4j:WARN No appenders could be found for logger (org.apache.kafka.connect.json.JsonConverterConfig).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

                  ===========================================
                  =        _  __ _____  ____  _             =
                  =       | |/ // ____|/ __ \| |            =
                  =       | ' /| (___ | |  | | |            =
                  =       |  <  \___ \| |  | | |            =
                  =       | . \ ____) | |__| | |____        =
                  =       |_|\_\_____/ \___\_\______|       =
                  =                                         =
                  =  Streaming SQL Engine for Apache Kafka® =
                  ===========================================

Copyright 2017-2018 Confluent Inc.

CLI v5.1.3, Server v5.1.3 located at http://localhost:8088

Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!

ksql>

To Reproduce Steps to reproduce the behavior, include:

  1. The version of KSQL. v5.1.3 with CentOS Linux release 7.5.1804 (Core)

  2. Sample source data.

  3. Any SQL statements you ran None

Expected behavior /etc/ksql/log4j-file.properties will be read by KSQL.

Actual behaviour /usr/bin/ksql works. In CentOS7, there is a symlink as below

# ls -lh /bin
lrwxrwxrwx. 1 root root 7 Dec 18  2018 /bin -> usr/bin

But /bin/ksql does not work. I think ksql command does not support when base_dir will become "/". Following error part's //etc/ksql/log4j-file.properties is strange. ("/" is duplicated)

log4j:ERROR Could not read configuration file from URL [file://etc/ksql/log4j-file.properties]

Can you check it? Thanks.

kingindanord commented 4 years ago

I have the same problem

barebu commented 4 years ago

I have the same problem

barebu commented 4 years ago

@yodo-t I solved this problem like this. On Centos 7, edited vim / usr / bin / ksql, removed "/" in the line "base_dir = $ (cd" $ (dirname "$ {BASH_SOURCE [0]}") / .. "&& pwd)"

#!/bin/bash
# (Copyright) [2017 - 2017] Confluent, Inc.

#
# Use shellcheck to lint this file
#

base_dir=$( cd "$( dirname "${BASH_SOURCE[0]}" ).." && pwd )

: "${KSQL_CONFIG_DIR:="$base_dir/config"}"

# logj4 settings
if [ -z "$KSQL_LOG4J_OPTS" ]; then
  # Test for files from dev -> packages so this will work as expected in dev if you have packages
  # installed
  if [ -e "$base_dir/config/log4j.properties" ]; then # Dev environment
    KSQL_CONFIG_DIR="$base_dir/config"
  elif [ -e "$base_dir/etc/ksql/log4j.properties" ]; then # Simple zip file layout
    KSQL_CONFIG_DIR="$base_dir/etc/ksql"
  elif [ -e "/etc/ksql/log4j.properties" ]; then # Normal install layout
    KSQL_CONFIG_DIR="/etc/ksql"
  fi
fi

: "${KSQL_LOG4J_OPTS:=""}"

# Use file logging by default
if [ -z "$KSQL_LOG4J_OPTS" ]; then
  export KSQL_LOG4J_OPTS="-Dlog4j.configuration=file:$KSQL_CONFIG_DIR/log4j-file.properties"
fi

exec "$base_dir"/bin/ksql-run-class io.confluent.ksql.Ksql "$@"
magrawalgit commented 2 years ago

@yodo-t I solved this problem like this. On Centos 7, edited vim / usr / bin / ksql, removed "/" in the line "base_dir = (cd" (dirname "$ {BASH_SOURCE [0]}") / .. "&& pwd)"

#!/bin/bash
# (Copyright) [2017 - 2017] Confluent, Inc.

#
# Use shellcheck to lint this file
#

base_dir=$( cd "$( dirname "${BASH_SOURCE[0]}" ).." && pwd )

: "${KSQL_CONFIG_DIR:="$base_dir/config"}"

# logj4 settings
if [ -z "$KSQL_LOG4J_OPTS" ]; then
  # Test for files from dev -> packages so this will work as expected in dev if you have packages
  # installed
  if [ -e "$base_dir/config/log4j.properties" ]; then # Dev environment
    KSQL_CONFIG_DIR="$base_dir/config"
  elif [ -e "$base_dir/etc/ksql/log4j.properties" ]; then # Simple zip file layout
    KSQL_CONFIG_DIR="$base_dir/etc/ksql"
  elif [ -e "/etc/ksql/log4j.properties" ]; then # Normal install layout
    KSQL_CONFIG_DIR="/etc/ksql"
  fi
fi

: "${KSQL_LOG4J_OPTS:=""}"

# Use file logging by default
if [ -z "$KSQL_LOG4J_OPTS" ]; then
  export KSQL_LOG4J_OPTS="-Dlog4j.configuration=file:$KSQL_CONFIG_DIR/log4j-file.properties"
fi

exec "$base_dir"/bin/ksql-run-class io.confluent.ksql.Ksql "$@"

This worked...Thanks!

felipeavilis commented 3 months ago

The solution mentioned above didn't work for me. Removing the slash "/" from line 8 coused another error:

bin/ksql: line 8: cd: /bin..: No such file or directory

So i added curly brackets { } to every base_dir variable and removed slash "/" after it.

#!/bin/bash
# (Copyright) [2017 - 2017] Confluent, Inc.

#
# Use shellcheck to lint this file
#

base_dir=$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )

: "${KSQL_CONFIG_DIR:="${base_dir}config"}"

# logj4 settings
if [ -z "$KSQL_LOG4J_OPTS" ]; then
  # Test for files from dev -> packages so this will work as expected in dev if you have packages
  # installed
  if [ -e "${base_dir}config/log4j.properties" ]; then # Dev environment
    KSQL_CONFIG_DIR="${base_dir}config"
  elif [ -e "${base_dir}etc/ksqldb/log4j.properties" ]; then # Simple zip file layout
    KSQL_CONFIG_DIR="${base_dir}etc/ksqldb"
  elif [ -e "/etc/ksqldb/log4j.properties" ]; then # Normal install layout
    KSQL_CONFIG_DIR="/etc/ksqldb"
  fi
fi

: "${KSQL_LOG4J_OPTS:=""}"

# Use file logging by default
if [ -z "$KSQL_LOG4J_OPTS" ]; then
  export KSQL_LOG4J_OPTS="-Dlog4j.configuration=file:$KSQL_CONFIG_DIR/log4j-file.properties"
fi

exec "${base_dir}"bin/ksql-run-class io.confluent.ksql.Ksql "$@"

And the result is... no errors

# ksql http://localhost:8088

                  ===========================================
                  =       _              _ ____  ____       =
                  =      | | _____  __ _| |  _ \| __ )      =
                  =      | |/ / __|/ _` | | | | |  _ \      =
                  =      |   <\__ \ (_| | | |_| | |_) |     =
                  =      |_|\_\___/\__, |_|____/|____/      =
                  =                   |_|                   =
                  =        The Database purpose-built       =
                  =        for stream processing apps       =
                  ===========================================

Copyright 2017-2022 Confluent Inc.

CLI v7.4.5, Server v7.4.5 located at http://localhost:8088
Server Status: RUNNING

Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!

ksql> exit
Exiting ksqlDB.
#
alfa7h commented 1 week ago

Thanks @felipeavilis your solution the most suitable for me