confluentinc / librdkafka

The Apache Kafka C/C++ library
Other
7.36k stars 3.11k forks source link

libzstd not available at build time #4698

Closed mcleantom closed 2 months ago

mcleantom commented 2 months ago

Description

I am trying to add zstd compression to my kafka messages, however I am getting the error

Failed to set compression codec: Unsupported value "zstd" for configuration property "compression.codec": libzstd not available at build time

I am installing librdkafka via vcpkg.

How can I add zstd to librdkafka when installing via vcpkg?

How to reproduce

Create a vcpkg.json file

{
  "name": "test-kafka-zstd",
  "dependencies": [
     "librdkafka",
  ]
}

Create a test script main.cpp

#incldue <librdkafka/rdkafka.h>

int main() {
    rd_kafka_conf_t* conf = rd_kafka_conf_new();
    char errstr[512];

    if (rd_kafka_conf_set(conf, "compression.codec", "zstd", errstr, sizeof(errstr)) != RD_KAFKA_CONF_OK) {
        std::cerr << "Failed to set compression codec: " << errstr << std::endl;
        return 1;
    }

    return 0;
}

Create a CMakeLists.txt file:

cmake_minimum_required(VERSION 3.10)
project(KafkaTest)

find_package(RdKafka CONFIG REQUIRED)

add_executable(test main.cpp)
target_link_libraries(test PRIVATE RdKafka::rdkafka++)

And build the executable:

mkdir build
cd build
cmake .. -DCMAKE_TOOLCHAIN_FILE=/path/to/vcpkg/script/buildsystems/vcpkg/cmake

And then run the file.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

Thanks for any help!

mcleantom commented 2 months ago

Just need to add features to my vcpkg.json file:

{
  "name": "test-kafka-zstd",
  "dependencies" [
        {
           "name": "librdkafka",
           "features": ["zstd"]
        }
    ]
}