Closed pigeon-fancier closed 1 year ago
需知
升级feapder,保证feapder是最新版,若BUG仍然存在,则详细描述问题
pip install --upgrade feapder
解析不了web
File "D:\application\code-env\python3.11\Lib\site-packages\feapder\core\parser_control.py", line 568, in deal_request results = parser.parse(request, response) │ │ │ └ <Response [200]> │ │ └ <Request https://m.baidu.com/> │ └ <function SpiderTest.parse at 0x000001ACC6F16DE0> └ <SpiderTest(Thread-1, started 16256)>
import feapder class SpiderTest(feapder.AirSpider): def start_requests(self): # for i in range(1, 15): yield feapder.Request("https://m.baidu.com/") def parse(self, request, response): response.encoding_errors = 'ignore' print(response.text) def parse_detail(self, request, response): """ 解析详情 """ # 取url url = request.url # 取title title = request.title # 解析正文 content = response.xpath( 'string(//div[@class="content"])' ).extract_first() # string 表达式是取某个标签下的文本,包括子标签文本 print("url", url) print("title", title) print("content", content) if __name__ == "__main__": SpiderTest().start()
3.11 用 feapder==1.8.6b6
pip3 install feapder==1.8.6b6
后面会发布正式版
需知
升级feapder,保证feapder是最新版,若BUG仍然存在,则详细描述问题
解析不了web
File "D:\application\code-env\python3.11\Lib\site-packages\feapder\core\parser_control.py", line 568, in deal_request results = parser.parse(request, response) │ │ │ └ <Response [200]> │ │ └ <Request https://m.baidu.com/> │ └ <function SpiderTest.parse at 0x000001ACC6F16DE0> └ <SpiderTest(Thread-1, started 16256)>