confluentinc / confluent-kafka-go

Confluent's Apache Kafka Golang client
Apache License 2.0
4.57k stars 651 forks source link

Consumer error: Decompression (codec 0x1) of message at 31439 of 841 bytes failed: Local: Not implemented (<nil>) #446

Open ayaya118 opened 4 years ago

ayaya118 commented 4 years ago

code: package main

import ( "fmt" "gopkg.in/confluentinc/confluent-kafka-go.v1/kafka" )

func main() { c, err := kafka.NewConsumer(&kafka.ConfigMap{ "bootstrap.servers": "192.168.5.32:9082", "group.id": "myGroup", "auto.offset.reset": "earliest", })

if err != nil {
    panic(err)
}

err =c.SubscribeTopics([]string{"liaobe_php_analyses_log_uat"}, nil)
fmt.Println(err)
for {
    msg, err := c.ReadMessage(-1)
    if err == nil {
        fmt.Printf("Message on %s: %s\n", msg.TopicPartition, string(msg.Value))
    } else {
        fmt.Printf("Consumer error: %v (%v)\n", err, msg)
    }
}

c.Close()

}

run:

[root@localhost dy]# go run main.go

Consumer error: Decompression (codec 0x1) of message at 31439 of 841 bytes failed: Local: Not implemented () Consumer error: Decompression (codec 0x1) of message at 31439 of 841 bytes failed: Local: Not implemented () Consumer error: Decompression (codec 0x1) of message at 31439 of 841 bytes failed: Local: Not implemented () Consumer error: Decompression (codec 0x1) of message at 31439 of 841 bytes failed: Local: Not implemented ()
edenhill commented 4 years ago

Codec 0x1 is GZIP, this would imply that your build of librdkafka lacks GZIP support because zlib was not found during the build of librdkafka (see librdkafka ./configure output).

edenhill commented 4 years ago

How did you install librdkafka? and what librdkafka version?

Oskar-PJ commented 2 years ago

Was any solution found for this problem? I'm getting a similar error on my windows server:

KafkaError{code=_NOT_IMPLEMENTED,val=-170,str="Decompression (codec 0x4) of message at 25330891 of 505 bytes failed: Local: Not implemented"}

I'm using a conda environment, librdkafka automatically installed with python-confluent-kafka. (conda install python-confluent-kafka)

python-confluent-kafka: 1.9.0 version: librdkafka 1.9.1

I've asked the same question here https://stackoverflow.com/questions/73298124/0x4-decompression-error-in-python-confluent-kafka

The thing is my everythin works fine on my local machine, but once I'm on the windows server I'm getting this error.

Using kafka (not python) from the cmd for testing purposes works both on the server and locally, e.g.:

kafka-console-consumer.bat --consumer.config path\to\kafkaconfig.properties --bootstrap-server server:1234 --topic topic.name --group a.test.group