search

confluentinc / confluent-kafka-python

Confluent's Kafka Python Client
http://docs.confluent.io/current/clients/confluent-kafka-python
Other
124 stars 896 forks source link

Unable to install confluent-kafka package in python3.13 #1802

Closed kumaranvpl closed 1 month ago

kumaranvpl commented 2 months ago

Description

I have tried to install confluent-kafka using the command pip install confluent-kafka with python version 3.13.0rc1. The installation fails with the following error:

In file included from /tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src/Admin.c:17:
      /tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error: librdkafka/rdkafka.h: No such file or directory

Python version 3.13.0rc1 is the penultimate preview release of python 3.13, with the stable version scheduled for release on October 1st, 2024, which is roughly 40 days from now. So, please add support for python 3.13 version. For more info regarding the release of the stable version of python 3.13, please checkout https://www.python.org/downloads/release/python-3130rc1/

How to reproduce

  1. Install python3.13
  2. Create and activate venv
  3. Try to install confluent-kafka using the command pip install confluent-kafka.

Checklist

Please provide the following information:

Error log

Collecting confluent-kafka
  Using cached confluent-kafka-2.5.0.tar.gz (136 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: confluent-kafka
  Building wheel for confluent-kafka (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for confluent-kafka (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [66 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-313
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka
      copying src/confluent_kafka/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka
      copying src/confluent_kafka/error.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka
      copying src/confluent_kafka/deserializing_consumer.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka
      copying src/confluent_kafka/serializing_producer.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/serialization
      copying src/confluent_kafka/serialization/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/serialization
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/_model
      copying src/confluent_kafka/_model/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/_model
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/avro
      copying src/confluent_kafka/avro/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/avro
      copying src/confluent_kafka/avro/error.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/avro
      copying src/confluent_kafka/avro/load.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/avro
      copying src/confluent_kafka/avro/cached_schema_registry_client.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/avro
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_listoffsets.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_acl.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_group.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_cluster.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_scram.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_config.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_topic.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_metadata.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_resource.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      copying src/confluent_kafka/admin/_records.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/admin
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      copying src/confluent_kafka/schema_registry/protobuf.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      copying src/confluent_kafka/schema_registry/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      copying src/confluent_kafka/schema_registry/schema_registry_client.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      copying src/confluent_kafka/schema_registry/error.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      copying src/confluent_kafka/schema_registry/json_schema.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      copying src/confluent_kafka/schema_registry/avro.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/schema_registry
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/_util
      copying src/confluent_kafka/_util/conversion_util.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/_util
      copying src/confluent_kafka/_util/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/_util
      copying src/confluent_kafka/_util/validation_util.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/_util
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/kafkatest
      copying src/confluent_kafka/kafkatest/verifiable_consumer.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/kafkatest
      copying src/confluent_kafka/kafkatest/verifiable_producer.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/kafkatest
      copying src/confluent_kafka/kafkatest/verifiable_client.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/kafkatest
      copying src/confluent_kafka/kafkatest/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/kafkatest
      creating build/lib.linux-x86_64-cpython-313/confluent_kafka/avro/serializer
      copying src/confluent_kafka/avro/serializer/__init__.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/avro/serializer
      copying src/confluent_kafka/avro/serializer/message_serializer.py -> build/lib.linux-x86_64-cpython-313/confluent_kafka/avro/serializer
      running build_ext
      building 'confluent_kafka.cimpl' extension
      creating build/temp.linux-x86_64-cpython-313
      creating build/temp.linux-x86_64-cpython-313/tmp
      creating build/temp.linux-x86_64-cpython-313/tmp/pip-install-8_h31mun
      creating build/temp.linux-x86_64-cpython-313/tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d
      creating build/temp.linux-x86_64-cpython-313/tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src
      creating build/temp.linux-x86_64-cpython-313/tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka
      creating build/temp.linux-x86_64-cpython-313/tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src
      x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/kumaran/dev/faststream/venv313/include -I/usr/include/python3.13 -c /tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src/Admin.c -o build/temp.linux-x86_64-cpython-313/tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src/Admin.o
      In file included from /tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src/Admin.c:17:
      /tmp/pip-install-8_h31mun/confluent-kafka_4106eda5da574398b605fda22ad32b9d/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error: librdkafka/rdkafka.h: No such file or directory
         23 | #include <librdkafka/rdkafka.h>
            |          ^~~~~~~~~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for confluent-kafka
Failed to build confluent-kafka
pranavrth commented 2 months ago

We don't provide Python 3.13 wheels so the installation will fallback to installation from source distribution. The source distribution expects librdkafka to be present which is not present in your case and hence facing this issue. Please refer this wiki for the installation in your case.

pranavrth commented 1 month ago

Published v2.6.0 with Python 3.13 wheels. Closing this issue. Please raise a new issue if you face any problem.