search

rmax / scrapy-redis

Redis-based components for Scrapy.
http://scrapy-redis.readthedocs.io
MIT License
5.54k stars 1.59k forks source link

Add Kafka Topic Integration to Scrapy Redis #288

Closed hamed21q closed 3 months ago

hamed21q commented 10 months ago

I suggest adding a feature to Scrapy Redis for seamless integration with Kafka, enabling users to publish crawled items directly to a Kafka topic. I'm willing to implement this feature and welcome feedback from the community. but this feature didn't exist in todos.rst so i opened this issue to ask if this feature is valuable for the maintainers or not.

best regurds

rmax commented 10 months ago

I'd suggest to create it as standalone library.