search

CrawlScript / WebCollector

WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
https://github.com/CrawlScript/WebCollector
GNU General Public License v3.0
3.07k stars 1.45k forks source link

depth太大可能导致OOM #79

Closed carryxyh closed 6 years ago

carryxyh commented 6 years ago

新增一个LOG，depth大于10时，打印日志，提醒使用方depth太大，可能导致OOM

carryxyh commented 6 years ago

问题分析：https://github.com/CrawlScript/WebCollector/issues/78