After sometime of crawling, the fronties crashes and throws an exception related to rethinkdb.
Stacktrace:
com.rethinkdb.gen.exc.ReqlClientError: Token 4612 not in stream cache.
frontier | at com.rethinkdb.ErrorBuilder.build(ErrorBuilder.java:79)
frontier | at com.rethinkdb.net.Response.makeError(Response.java:171)
frontier | at com.rethinkdb.net.Connection.runQuery(Connection.java:288)
frontier | at com.rethinkdb.net.Connection.runQuery(Connection.java:253)
frontier | at com.rethinkdb.net.Connection.runQuery(Connection.java:249)
frontier | at com.rethinkdb.net.Connection.stop(Connection.java:348)
frontier | at com.rethinkdb.net.Cursor.close(Cursor.java:55)
frontier | at org.aksw.simba.squirrel.queue.RDBQueue.queueContainsIpAddressTypeKey(RDBQueue.java:101)
frontier | at org.aksw.simba.squirrel.queue.RDBQueue.addToQueue(RDBQueue.java:85)
frontier | at org.aksw.simba.squirrel.queue.AbstractIpAddressBasedQueue.addUri(AbstractIpAddressBasedQueue.java:41)
frontier | at org.aksw.simba.squirrel.frontier.impl.FrontierImpl.addNewUri(FrontierImpl.java:119)
frontier | at org.aksw.simba.squirrel.frontier.impl.FrontierImpl.addNewUris(FrontierImpl.java:102)
frontier | at org.aksw.simba.squirrel.components.FrontierComponent.handleData(FrontierComponent.java:132)
frontier | at org.aksw.simba.squirrel.components.FrontierComponent.handleData(FrontierComponent.java:114)
frontier | at org.aksw.simba.squirrel.rabbit.RPCServer$MsgProcessingTask.run(RPCServer.java:72)
frontier | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
frontier | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
frontier | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
frontier | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
frontier | at java.lang.Thread.run(Thread.java:748)
gsjunior86
commented
6 years ago
After sometime of crawling, the fronties crashes and throws an exception related to rethinkdb.
Stacktrace: