search

domainaware / parsedmarc

A Python package and CLI for parsing aggregate and forensic DMARC reports
https://domainaware.github.io/parsedmarc/
Apache License 2.0
961 stars 209 forks source link

container with py3.12 fails to run due to kafka-python being an old version, kafka-python-ng replaces it #499

Open fsvm88 opened 3 months ago

fsvm88 commented 3 months ago

Hello,

I was updating my docker images, and switched the base image python 3.12.

The build runs, but the latest available kafka-python module does not have some py3.12 compat fixes, which leads it to fail at execution with the following:

Traceback (most recent call last):
  File "/usr/local/bin/parsedmarc", line 5, in <module>
    from parsedmarc.cli import _main
  File "/usr/local/lib/python3.12/site-packages/parsedmarc/cli.py", line 19, in <module>
    from parsedmarc import get_dmarc_reports_from_mailbox, watch_inbox, \
  File "/usr/local/lib/python3.12/site-packages/parsedmarc/kafkaclient.py", line 6, in <module>
    from kafka import KafkaProducer
  File "/usr/local/lib/python3.12/site-packages/kafka/__init__.py", line 23, in <module>
    from kafka.consumer import KafkaConsumer
  File "/usr/local/lib/python3.12/site-packages/kafka/consumer/__init__.py", line 3, in <module>
    from kafka.consumer.group import KafkaConsumer
  File "/usr/local/lib/python3.12/site-packages/kafka/consumer/group.py", line 13, in <module>
    from kafka.consumer.fetcher import Fetcher
  File "/usr/local/lib/python3.12/site-packages/kafka/consumer/fetcher.py", line 19, in <module>
    from kafka.record import MemoryRecords
  File "/usr/local/lib/python3.12/site-packages/kafka/record/__init__.py", line 1, in <module>
    from kafka.record.memory_records import MemoryRecords, MemoryRecordsBuilder
  File "/usr/local/lib/python3.12/site-packages/kafka/record/memory_records.py", line 27, in <module>
    from kafka.record.legacy_records import LegacyRecordBatch, LegacyRecordBatchBuilder
  File "/usr/local/lib/python3.12/site-packages/kafka/record/legacy_records.py", line 50, in <module>
    from kafka.codec import (
  File "/usr/local/lib/python3.12/site-packages/kafka/codec.py", line 9, in <module>
    from kafka.vendor.six.moves import range
ModuleNotFoundError: No module named 'kafka.vendor.six.moves'

Investigating a little, I found out that the current maintainer of kafka-python is not the owner, and cannot publish updates to the kafka-python namespace on pypi anymore. He created a fork named kafka-python-ng telling people to switch over.

Simply switching to the fork resolves the runtime issue, and since it's the same codebase I'd expect it to work with little issues, but I did not explicitly test it as I don't have a setup for it.

davidande commented 2 months ago

same issue on Webtop (ubuntu-kde) trying to setup Parsedmarc failed. kafka-python-ng solved it.