search

rmax / scrapy-redis

Redis-based components for Scrapy.
http://scrapy-redis.readthedocs.io
MIT License
5.49k stars 1.59k forks source link

Add Kafka Topic Integration to Scrapy Redis #288

Open hamed21q opened 5 months ago

hamed21q commented 5 months ago

I suggest adding a feature to Scrapy Redis for seamless integration with Kafka, enabling users to publish crawled items directly to a Kafka topic. I'm willing to implement this feature and welcome feedback from the community. but this feature didn't exist in todos.rst so i opened this issue to ask if this feature is valuable for the maintainers or not.

best regurds

rmax commented 5 months ago

I'd suggest to create it as standalone library.