FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.
We used to always create topics automatically and provided no way to make it configurable. This PR introduces a new variable called allow_auto_create_topics to make automatic topic creation configurable.
I have included a warning when allow_auto_create_topics is set to False because confluent-kafka-python backfires/crumbles terribly if it tries to access an non existent topic as mentioned here - https://github.com/confluentinc/confluent-kafka-python/issues/1547.
There were few blocking calls in the __init__ method of Async Confluent Producer and Consumer. I have used run_in_executor to make them non-blocking.
Fixes #1486, #1363
Type of change
Please delete options that are not relevant.
[ ] Documentation (typos, code examples, or any documentation updates)
[x] Bug fix (a non-breaking change that resolves an issue)
[ ] New feature (a non-breaking change that adds functionality)
[ ] Breaking change (a fix or feature that would disrupt existing functionality)
[ ] This change requires a documentation update
Checklist
[x] My code adheres to the style guidelines of this project (scripts/lint.sh shows no errors)
[x] I have conducted a self-review of my own code
[ ] I have made the necessary changes to the documentation
[ ] My changes do not generate any new warnings
[ ] I have added tests to validate the effectiveness of my fix or the functionality of my new feature
[x] Both new and existing unit tests pass successfully on my local environment by running scripts/test-cov.sh
[x] I have ensured that static analysis tests are passing by running scripts/static-analysis.sh
[ ] I have included code examples to illustrate the modifications
Description
We used to always create topics automatically and provided no way to make it configurable. This PR introduces a new variable called
allow_auto_create_topicsto make automatic topic creation configurable. I have included a warning whenallow_auto_create_topicsis set toFalsebecauseconfluent-kafka-pythonbackfires/crumbles terribly if it tries to access an non existent topic as mentioned here - https://github.com/confluentinc/confluent-kafka-python/issues/1547. There were few blocking calls in the__init__method of Async Confluent Producer and Consumer. I have usedrun_in_executorto make them non-blocking.Fixes #1486, #1363
Type of change
Please delete options that are not relevant.
Checklist
scripts/lint.shshows no errors)scripts/test-cov.shscripts/static-analysis.sh