CrawlScript / WebCollector

WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup a multi-threaded web crawler in less than 5 minutes.
https://github.com/CrawlScript/WebCollector
GNU General Public License v3.0
3.07k stars 1.45k forks source link

你好! LockTimeoutException #92

Closed simplecnst closed 6 years ago

simplecnst commented 6 years ago

你好,报这个错,怎么解决呢? com.sleepycat.je.LockTimeoutException: (JE 5.0.73) Lock expired. Locker 264328825 -1_Thread-7381_ThreadLocker: waited for lock on database=_jeNameMap LockAddr:556941658 LSN=0x0/0x9ef type=WRITE grant=WAIT_PROMOTION timeoutMillis=500 startTime=1535089847412 endTime=1535089847912 Owners: [, ] Waiters: [] Transaction 264328825 -1_Thread-7381_ThreadLocker owns LockAddr:556941658 Transaction 264328825 -1_Thread-7381_ThreadLocker waits for LockAddr:556941658

at com.sleepycat.je.txn.LockManager.newLockTimeoutException(LockManager.java:664) at com.sleepycat.je.txn.LockManager.makeTimeoutMsgInternal(LockManager.java:623) at com.sleepycat.je.txn.SyncedLockManager.makeTimeoutMsg(SyncedLockManager.java:97) at com.sleepycat.je.txn.LockManager.lockInternal(LockManager.java:390) at com.sleepycat.je.txn.LockManager.lock(LockManager.java:276) at com.sleepycat.je.txn.BasicLocker.lockInternal(BasicLocker.java:118) at com.sleepycat.je.txn.Locker.lock(Locker.java:443) at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:2621) at com.sleepycat.je.dbi.CursorImpl.lockLN(CursorImpl.java:2422) at com.sleepycat.je.dbi.CursorImpl.searchAndPosition(CursorImpl.java:2150) at com.sleepycat.je.dbi.CursorImpl.searchAndPosition(CursorImpl.java:2020) at com.sleepycat.je.dbi.DbTree.lockNameLN(DbTree.java:880) at com.sleepycat.je.dbi.DbTree.doRemoveDb(DbTree.java:1099) at com.sleepycat.je.dbi.DbTree.dbRemove(DbTree.java:1152) at com.sleepycat.je.Environment.removeDatabase(Environment.java:963) at cn.edu.hfut.dmic.webcollector.plugin.berkeley.BerkeleyDBManager.merge(BerkeleyDBManager.java:197) at cn.edu.hfut.dmic.webcollector.fetcher.Fetcher.fetchAll(Fetcher.java:305) at cn.edu.hfut.dmic.webcollector.crawler.Crawler.start(Crawler.java:136) at com.app.MatchURLThread.run(MatchURLThread.java:90)

hujunxianligong commented 6 years ago

已经添加了RocksDB内核,可以试试 import cn.edu.hfut.dmic.webcollector.plugin.rocks.BreadthCrawler;