redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
https://redpanda.com
9.73k stars 591 forks source link

AdminClient/describeLogDirs returns "The broker does not support DESCRIBE_LOG_DIRS" #903

Closed d-t-w closed 3 years ago

d-t-w commented 3 years ago

Kafka Client Library: 2.7.0

Our product uses the describeLogDirs method in the Kafka Admin Client API to inspect the disk consumed by topics.

/**
     * Query the information of all log directories on the given set of brokers
     * <p>
     * This is a convenience method for {@link #describeLogDirs(Collection, DescribeLogDirsOptions)} with default options.
     * See the overload for more details.
     * <p>
     * This operation is supported by brokers with version 1.0.0 or higher.
     *
     * @param brokers A list of brokers
     * @return The DescribeLogDirsResult
     */
    default DescribeLogDirsResult describeLogDirs(Collection<Integer> brokers) {
        return describeLogDirs(brokers, new DescribeLogDirsOptions());
    }

When we use the method with the Dockerized RedPanda we get the following error:

16:02:05.094 ERROR [async-thread-macro-1] operatr.observation – observation error: {:via [{:type java.util.concurrent.ExecutionException, :message org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_LOG_DIRS, :at [org.apache.kafka.common.internals.KafkaFutureImpl wrapAndThrow KafkaFutureImpl.java 45]} {:type org.apache.kafka.common.errors.UnsupportedVersionException, :message The broker does not support DESCRIBE_LOG_DIRS}], :cause The broker does not support DESCRIBE_LOG_DIRS, :id {:id red-panda, :domain :cluster}, :action :observe, :command {:type :kafka/broker, :broker-ids (1)}}

We are using the deprecated .all method on the returned DescribeLogDirsResult, but calling any method on that result causes the same error.

dev notes:

yeah, (.describeLogDirs client [(int 1)]) returns something
but any method (.allDescriptions, .descriptions,  all  etc)
throw org.apache.kafka.common.errors.UnsupportedVersionException: The broker does not support DESCRIBE_LOG_DIRS
with kafka 2.7.0 client
emaxerrno commented 3 years ago

thanks Derek! much appreciated.

team for context: https://docs.kpow.io/configuration/redpanda uses the admin describe-log-dirs api.

dotnwat commented 3 years ago

Thanks @d-t-w can take care of this in a snap

d-t-w commented 3 years ago

Awesome @dotnwat, thanks!