search

Insutanto / scrapy-distributed

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
55 stars 11 forks source link

dynamic web crawlers #10

Open vijaygao opened 3 years ago

vijaygao commented 3 years ago

Can you add dynamic web crawlers into your project? Need to use simulation click technology and anti-climbing mechanism. Thx！

Insutanto commented 3 years ago

Maybe, those will be implemented in the future. I am glad to support dynamic web crawlers.