search

neo4j-contrib / authentication-extension

Neo4j Server Auth Extension
72 stars 28 forks source link

javax.transaction.SystemException: TM has encountered some problem #21

Closed kwent closed 10 years ago

kwent commented 10 years ago

Hi there,

We put the authentication-extension in our neo4j server and after one hour of running, calling the rest API raised this exception: 

javax.transaction.SystemException: TM has encountered some problem, please perform neccesary action (tx recovery/restart)

The only way to avoid this is to restart manually the neo4j server so far.

I already read this a similar problem but it doesn't help for our case :

Any ideas ?

jexp commented 10 years ago

Can you provide the full stack trace? And the versions you use?

kwent commented 10 years ago

I have no more information just this. Version : neo4j-enterprise-1.9.5 hosted on Rackspace.

jexp commented 10 years ago

Can you check your servers logs? Esp. data/log and data/graph.db/messages.log for error messages.

And if you use enterprise and are a customer probably raising a support ticket makes sense.

kwent commented 10 years ago

Here the backtrace but it's weird cause new relic is telling me that org.neo4j.server.extension. auth. AuthenticationFilter raised the exception but the backtrace sounds not telling the same thing.

Jan 29, 2014 6:07:56 PM com.sun.jersey.spi.container.ContainerResponse mapMappableContainerException
SEVERE: The RuntimeException could not be mapped to a response, re-throwing to the HTTP container
java.lang.RuntimeException: javax.transaction.SystemException: TM has encountered some problem, please perform neccesary action (tx recovery/restart)
    at org.neo4j.kernel.impl.transaction.TxManager.getTransactionState(TxManager.java:954)
    at org.neo4j.kernel.impl.core.NodeManager.getTransactionState(NodeManager.java:1321)
    at org.neo4j.kernel.impl.core.Primitive.getProperty(Primitive.java:154)
    at org.neo4j.kernel.impl.core.NodeImpl.getProperty(NodeImpl.java:53)
    at org.neo4j.kernel.impl.core.NodeProxy.getProperty(NodeProxy.java:155)
    at org.neo4j.example.unmanagedextension.Queries.addLocationCreatePerson(Queries.java:286)
    at org.neo4j.example.unmanagedextension.Queries.findFofs3(Queries.java:405)
    at org.neo4j.example.unmanagedextension.Queries.getFOFResults3(Queries.java:356)
    at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:322)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: javax.transaction.SystemException: TM has encountered some problem, please perform neccesary action (tx recovery/restart)
    at org.neo4j.kernel.impl.transaction.TxManager.assertTmOk(TxManager.java:286)
    at org.neo4j.kernel.impl.transaction.TxManager.getTransaction(TxManager.java:656)
    at org.neo4j.kernel.impl.transaction.TxManager.getTransactionState(TxManager.java:950)
    ... 43 more
Caused by: javax.transaction.xa.XAException
    at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:576)
    at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:394)
    at org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:344)
    at org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:134)
    at org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:127)
    at org.neo4j.server.rest.web.DatabaseActions.addToRelationshipIndex(DatabaseActions.java:707)
    at org.neo4j.server.rest.web.RestfulGraphDatabase.addToRelationshipIndex(RestfulGraphDatabase.java:895)
    at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
    at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
    at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
    at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
    at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:322)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    ... 3 more
Caused by: java.lang.RuntimeException: java.io.IOException: Map failed
    at org.neo4j.index.impl.lucene.LuceneDataSource.refreshSearcher(LuceneDataSource.java:514)
    at org.neo4j.index.impl.lucene.LuceneDataSource.refreshSearcherIfNeeded(LuceneDataSource.java:631)
    at org.neo4j.index.impl.lucene.LuceneDataSource.getIndexSearcher(LuceneDataSource.java:575)
    at org.neo4j.index.impl.lucene.CommitContext.ensureWriterInstantiated(CommitContext.java:58)
    at org.neo4j.index.impl.lucene.LuceneCommand$AddRelationshipCommand.perform(LuceneCommand.java:225)
    at org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:286)
    at org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:322)
    at org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:489)
    at org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64)
    at org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:570)
    ... 42 more
Caused by: java.io.IOException: Map failed
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:748)
    at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:270)
    at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:220)
    at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:345)
    at org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:231)
    at org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:201)
    at org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:604)
    at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3587)
    at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3552)
    at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:450)
    at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:399)
    at org.apache.lucene.index.DirectoryReader.doOpenFromWriter(DirectoryReader.java:413)
    at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:432)
    at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:375)
    at org.apache.lucene.index.DirectoryReader.doOpenIfChanged(DirectoryReader.java:392)
    at org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:634)
    at org.neo4j.index.impl.lucene.LuceneDataSource.refreshSearcher(LuceneDataSource.java:503)
    ... 51 more
Caused by: java.lang.OutOfMemoryError: Map failed
    at sun.nio.ch.FileChannelImpl.map0(Native Method)
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:745)
    ... 67 more
jexp commented 10 years ago

Pretty obvious and unrelated to both extensions (perhaps).

You create a transaction that's too large and you get an Out Of Memory error.