search

tkuhn / nanopub-server

A simple server to publish nanopublications
MIT License
12 stars 11 forks source link

Server crashes when collecting large numbers of nanopubs (Too many open files) #1

Closed antonisloizou closed 9 years ago

antonisloizou commented 9 years ago

Attempting to sync with 5M+ nanopubs (using build from commit 938264cc569c8d0e72cc6df535de1de0954c89a7)

catalina.out

20:50:57,315  INFO CollectNanopubs:110 - Process page 3251 from http://s1.semanticscience.org:8082/
Exception in thread "Thread-3" java.lang.NoClassDefFoundError: org/apache/http/impl/client/RequestWrapper
        at org.apache.http.impl.client.DefaultHttpRequestRetryHandler.requestIsAborted(DefaultHttpRequestRetryHandler.java:197)
        at org.apache.http.impl.client.DefaultHttpRequestRetryHandler.retryRequest(DefaultHttpRequestRetryHandler.java:149)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:92)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
        at org.nanopub.extra.server.NanopubServerUtils.loadList(NanopubServerUtils.java:40)
        at org.nanopub.extra.server.NanopubServerUtils.loadNanopubUriList(NanopubServerUtils.java:29)
        at org.nanopub.extra.server.NanopubServerUtils.loadNanopubUriList(NanopubServerUtils.java:33)
        at ch.tkuhn.nanopub.server.CollectNanopubs.processPage(CollectNanopubs.java:115)
        at ch.tkuhn.nanopub.server.CollectNanopubs.run(CollectNanopubs.java:88)
        at ch.tkuhn.nanopub.server.ScanPeers.collectNanopubs(ScanPeers.java:141)
        at ch.tkuhn.nanopub.server.ScanPeers.collectAndContactPeers(ScanPeers.java:92)
        at ch.tkuhn.nanopub.server.ScanPeers.run(ScanPeers.java:65)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.ClassNotFoundException: org.apache.http.impl.client.RequestWrapper
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1680)
        at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
        ... 16 more
Jan 28, 2015 8:50:57 PM org.apache.tomcat.util.net.JIoEndpoint$Acceptor run
SEVERE: Socket accept failed
java.net.SocketException: Too many open files
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
        at java.net.ServerSocket.accept(ServerSocket.java:498)
        at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
        at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:352)
        at java.lang.Thread.run(Thread.java:744)
tkuhn commented 9 years ago

I haven't seen this with my servers. I need to investigate. Thanks for reporting!

tkuhn commented 9 years ago

This should be fixed with the latest commit e8a5c9511cd061c5f2cb77c6114d2430ffc2c8cd. Now all streams are properly closed. It's difficult to verify that the problem has really disappeared though, as it is not easy to reproduce. We will see...