Closed mcleantom closed 2 months ago
I am trying to add zstd compression to my kafka messages, however I am getting the error
Failed to set compression codec: Unsupported value "zstd" for configuration property "compression.codec": libzstd not available at build time
I am installing librdkafka via vcpkg.
How can I add zstd to librdkafka when installing via vcpkg?
Create a vcpkg.json file
vcpkg.json
{ "name": "test-kafka-zstd", "dependencies": [ "librdkafka", ] }
Create a test script main.cpp
main.cpp
#incldue <librdkafka/rdkafka.h> int main() { rd_kafka_conf_t* conf = rd_kafka_conf_new(); char errstr[512]; if (rd_kafka_conf_set(conf, "compression.codec", "zstd", errstr, sizeof(errstr)) != RD_KAFKA_CONF_OK) { std::cerr << "Failed to set compression codec: " << errstr << std::endl; return 1; } return 0; }
Create a CMakeLists.txt file:
CMakeLists.txt
cmake_minimum_required(VERSION 3.10) project(KafkaTest) find_package(RdKafka CONFIG REQUIRED) add_executable(test main.cpp) target_link_libraries(test PRIVATE RdKafka::rdkafka++)
And build the executable:
mkdir build cd build cmake .. -DCMAKE_TOOLCHAIN_FILE=/path/to/vcpkg/script/buildsystems/vcpkg/cmake
And then run the file.
IMPORTANT: We will close issues where the checklist has not been completed.
Please provide the following information:
debug=..
Thanks for any help!
Just need to add features to my vcpkg.json file:
features
{ "name": "test-kafka-zstd", "dependencies" [ { "name": "librdkafka", "features": ["zstd"] } ] }
Description
I am trying to add zstd compression to my kafka messages, however I am getting the error
I am installing librdkafka via vcpkg.
How can I add zstd to librdkafka when installing via vcpkg?
How to reproduce
Create a
vcpkg.json
fileCreate a test script
main.cpp
Create a
CMakeLists.txt
file:And build the executable:
And then run the file.
Checklist
IMPORTANT: We will close issues where the checklist has not been completed.
Please provide the following information:
debug=..
as necessary) from librdkafka: N/AThanks for any help!