search

howie6879 / ruia

Async Python 3.6+ web scraping micro-framework based on asyncio
https://www.howie6879.com/ruia/
Apache License 2.0
1.75k stars 181 forks source link

ruia 怎么使用伪造ip或者使用ip池 #117

Closed shuqian2017 closed 4 years ago

shuqian2017 commented 4 years ago

原因： 在使用的过程中，爬取的过程中容易导致IP地址被封，从而无法访问

howie6879 commented 4 years ago

https://github.com/howie6879/ruia/blob/master/examples/simple_spider/douban_spider.py

在aiohttp_kwargs参数引入你的ip就行：比如：aiohttp_kwargs = {"proxy": "http://xxx.com"}