CrawlScript / WebCollector

WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
https://github.com/CrawlScript/WebCollector
GNU General Public License v3.0
3.07k stars 1.45k forks source link

out of memory 问题。 #120

Open wangqifan opened 4 years ago

wangqifan commented 4 years ago

java堆内存超了。设置的是4G jmap查看系统有大量的com.sleepycat.je.tree.LN 对象。 1981583078135_ pic