Open dimsoul opened 8 months ago
我查阅爬虫参数支持设置post请求及参数post-data,但是实际设置了post-data的内容,每次请求都是超时,且我用wireshark抓包并没有抓到post请求报文;虽然最终的爬虫结果中确实收集了这个post请求,但是确实没有发包,请问是为什么?以下是爬虫的日志
time="2024-03-11T14:55:23+08:00" level=info msg="Init crawler task, host: testphp.vulnweb.com, max tab count: 5, max crawl count: 200, max runtime: 3600s" time="2024-03-11T14:55:23+08:00" level=info msg="filter mode: smart" time="2024-03-11T14:55:23+08:00" level=info msg="If no matches, default form input text: Crawlergo" time="2024-03-11T14:55:23+08:00" level=info msg="Start crawling." time="2024-03-11T14:55:23+08:00" level=info msg="filter repeat, target count: 2" time="2024-03-11T14:55:23+08:00" level=info msg="Crawling POST https://testphp.vulnweb.com/" time="2024-03-11T14:55:23+08:00" level=info msg="Crawling POST http://testphp.vulnweb.com/" time="2024-03-11T14:55:24+08:00" level=debug msg="main navigation req: http://testphp.vulnweb.com/" time="2024-03-11T14:55:24+08:00" level=debug msg="main navigation req: https://testphp.vulnweb.com/" time="2024-03-11T14:55:28+08:00" level=warning msg="navigate timeout http://testphp.vulnweb.com/"
我查阅爬虫参数支持设置post请求及参数post-data,但是实际设置了post-data的内容,每次请求都是超时,且我用wireshark抓包并没有抓到post请求报文;虽然最终的爬虫结果中确实收集了这个post请求,但是确实没有发包,请问是为什么?以下是爬虫的日志
time="2024-03-11T14:55:23+08:00" level=info msg="Init crawler task, host: testphp.vulnweb.com, max tab count: 5, max crawl count: 200, max runtime: 3600s" time="2024-03-11T14:55:23+08:00" level=info msg="filter mode: smart" time="2024-03-11T14:55:23+08:00" level=info msg="If no matches, default form input text: Crawlergo" time="2024-03-11T14:55:23+08:00" level=info msg="Start crawling." time="2024-03-11T14:55:23+08:00" level=info msg="filter repeat, target count: 2" time="2024-03-11T14:55:23+08:00" level=info msg="Crawling POST https://testphp.vulnweb.com/" time="2024-03-11T14:55:23+08:00" level=info msg="Crawling POST http://testphp.vulnweb.com/" time="2024-03-11T14:55:24+08:00" level=debug msg="main navigation req: http://testphp.vulnweb.com/" time="2024-03-11T14:55:24+08:00" level=debug msg="main navigation req: https://testphp.vulnweb.com/" time="2024-03-11T14:55:28+08:00" level=warning msg="navigate timeout http://testphp.vulnweb.com/"