Open sshzhang opened 6 years ago
Without having much more details, no one can help you.
Please share your debug.log, neo4j.conf What kind of operations have you been running?
You are also running a very old version, it would be good if you could upgrade to the most recent 3.4.4
Ok, Thank you
Here is the detail information
2018-07-21 09:42:05.944+0000 ERROR [o.n.b.v.r.ErrorReporter] Client triggered an unexpected error [TransactionStartFailed]: Database has encountered some problem, please perform necessary action (tx recovery/restart), reference ae963963-b933-45c9-8625-98ad22ed6cc8. Database has encountered some problem, please perform necessary action (tx recovery/restart) org.neo4j.graphdb.TransactionFailureException: Database has encountered some problem, please perform necessary action (tx recovery/restart) at org.neo4j.kernel.impl.factory.ClassicCoreSPI.beginTransaction(ClassicCoreSPI.java:182) at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransactionInternal(GraphDatabaseFacade.java:583) at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.beginTransaction(GraphDatabaseFacade.java:384) at org.neo4j.bolt.v1.runtime.TransactionStateMachineSPI.beginTransaction(TransactionStateMachineSPI.java:101) at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$1.run(TransactionStateMachine.java:235) at org.neo4j.bolt.v1.runtime.TransactionStateMachine.run(TransactionStateMachine.java:78) at org.neo4j.bolt.v1.runtime.BoltStateMachine$State$2.run(BoltStateMachine.java:405) at org.neo4j.bolt.v1.runtime.BoltStateMachine.run(BoltStateMachine.java:194) at org.neo4j.bolt.v1.messaging.BoltMessageRouter.lambda$onRun$3(BoltMessageRouter.java:80) at org.neo4j.bolt.v1.runtime.concurrent.RunnableBoltWorker.execute(RunnableBoltWorker.java:148) at org.neo4j.bolt.v1.runtime.concurrent.RunnableBoltWorker.run(RunnableBoltWorker.java:104) at java.lang.Thread.run(Thread.java:748) Caused by: org.neo4j.kernel.api.exceptions.TransactionFailureException: Database has encountered some problem, please perform necessary action (tx recovery/restart) at sun.reflect.GeneratedConstructorAccessor43.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.neo4j.kernel.internal.DatabaseHealth.assertHealthy(DatabaseHealth.java:62) at org.neo4j.kernel.impl.api.Kernel.newTransaction(Kernel.java:90) at org.neo4j.kernel.impl.factory.ClassicCoreSPI.beginTransaction(ClassicCoreSPI.java:174) ... 11 more Caused by: org.neo4j.kernel.impl.store.UnderlyingStorageException: Error performing check point at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.constructCombinedFailure(CheckPointScheduler.java:100) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:81) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:109) Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: Failed to flush at org.neo4j.kernel.impl.store.NeoStores.flush(NeoStores.java:253) at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:447) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64) ... 8 more Caused by: java.io.FileNotFoundException: /home/xiujiezhang/download/neo4j-community-3.2.11/data/databases/graph.db/neostore.counts.db.a (打开的文件过多) at java.io.RandomAccessFile.open0(Native Method) at java.io.RandomAccessFile.open(RandomAccessFile.java:316) at java.io.RandomAccessFile.(RandomAccessFile.java:243) at org.neo4j.io.fs.DefaultFileSystemAbstraction.open(DefaultFileSystemAbstraction.java:68) at org.neo4j.io.fs.DefaultFileSystemAbstraction.open(DefaultFileSystemAbstraction.java:53) at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.(SingleFilePageSwapper.java:152) at org.neo4j.io.pagecache.impl.SingleFilePageSwapperFactory.createPageSwapper(SingleFilePageSwapperFactory.java:71) at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.(MuninnPagedFile.java:137) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCache.map(MuninnPageCache.java:368) at org.neo4j.kernel.impl.store.kvstore.KeyValueWriter$OpeningStreamWriter.open(KeyValueWriter.java:316) at org.neo4j.kernel.impl.store.kvstore.KeyValueWriter.openStoreFile(KeyValueWriter.java:108) at org.neo4j.kernel.impl.store.kvstore.KeyValueStoreFileFormat.create(KeyValueStoreFileFormat.java:206) at org.neo4j.kernel.impl.store.kvstore.KeyValueStoreFileFormat.createStore(KeyValueStoreFileFormat.java:59) at org.neo4j.kernel.impl.store.kvstore.RotationStrategy.next(RotationStrategy.java:106) at org.neo4j.kernel.impl.store.kvstore.RotationState$Rotation.rotate(RotationState.java:92) at org.neo4j.kernel.impl.store.kvstore.RotationState$Rotation.rotate(RotationState.java:52) at org.neo4j.kernel.impl.store.kvstore.AbstractKeyValueStore$RotationTask.rotate(AbstractKeyValueStore.java:316) at org.neo4j.kernel.impl.store.kvstore.AbstractKeyValueStore$RotationTask.rotate(AbstractKeyValueStore.java:293) at org.neo4j.kernel.impl.store.counts.CountsTracker.rotate(CountsTracker.java:126) at org.neo4j.kernel.impl.store.NeoStores.flush(NeoStores.java:247) ... 12 more Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: Failed to flush at org.neo4j.kernel.impl.store.NeoStores.flush(NeoStores.java:253) at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:447) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64) ... 8 more Caused by: java.io.FileNotFoundException: /home/xiujiezhang/download/neo4j-community-3.2.11/data/databases/graph.db/neostore.counts.db.a (打开的文件过多) at java.io.RandomAccessFile.open0(Native Method) at java.io.RandomAccessFile.open(RandomAccessFile.java:316) at java.io.RandomAccessFile.(RandomAccessFile.java:243) at org.neo4j.io.fs.DefaultFileSystemAbstraction.open(DefaultFileSystemAbstraction.java:68) at org.neo4j.io.fs.DefaultFileSystemAbstraction.open(DefaultFileSystemAbstraction.java:53) at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.(SingleFilePageSwapper.java:152) at org.neo4j.io.pagecache.impl.SingleFilePageSwapperFactory.createPageSwapper(SingleFilePageSwapperFactory.java:71) at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.(MuninnPagedFile.java:137) at org.neo4j.io.pagecache.impl.muninn.MuninnPageCache.map(MuninnPageCache.java:368) at org.neo4j.kernel.impl.store.kvstore.KeyValueWriter$OpeningStreamWriter.open(KeyValueWriter.java:316) at org.neo4j.kernel.impl.store.kvstore.KeyValueWriter.openStoreFile(KeyValueWriter.java:108) at org.neo4j.kernel.impl.store.kvstore.KeyValueStoreFileFormat.create(KeyValueStoreFileFormat.java:206) at org.neo4j.kernel.impl.store.kvstore.KeyValueStoreFileFormat.createStore(KeyValueStoreFileFormat.java:59) at org.neo4j.kernel.impl.store.kvstore.RotationStrategy.next(RotationStrategy.java:106) at org.neo4j.kernel.impl.store.kvstore.RotationState$Rotation.rotate(RotationState.java:92) at org.neo4j.kernel.impl.store.kvstore.RotationState$Rotation.rotate(RotationState.java:52)
Here is the neo4j.conf file content
#
LOAD CSV
import files to be under the import
directory. Remove or comment it out toLOAD CSV
section of the manual for details.dbms.directories.import=import
dbms.connectors.default_listen_address=0.0.0.0
dbms.connector.bolt.enabled=true
dbms.connector.http.enabled=true
dbms.connector.https.enabled=true
#
#
#
#
#
#
#
#
#
LOAD CSV
. Setting this value to false
will cause Neo4j to fail LOAD CSV
dbms.tx_log.rotation.retention_policy=1 days
dbms.jvm.additional=-XX:+UseG1GC
dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow
initmemory
is not only allocated, but committed todbms.jvm.additional=-XX:+AlwaysPreTouch
dbms.jvm.additional=-XX:+UnlockExperimentalVMOptions dbms.jvm.additional=-XX:+TrustFinalNonStaticFields
dbms.jvm.additional=-XX:+DisableExplicitGC
dbms.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048
dbms.jvm.additional=-Djdk.tls.rejectClientInitiatedRenegotiation=true
dbms.windows_service_name=neo4j
dbms.jvm.additional=-Dunsupported.dbms.udc.source=tarball
@sshzhang It seems that you have opened too many file handles, maybe you can modify the limitation by 'ulimit -n' to work around the problem.
Yes it could very well be a file handles limitation issue. Did you try to increase that limit? On another note version 4.0 will have a rewritten counts store which doesn't have the problem of switching between a/b files like that.
Problem description: after run some times,The error has occurred.
Caused by: org.neo4j.kernel.api.exceptions.TransactionFailureException: Database has encountered some problem, please perform necessary action (tx recovery/restart) at sun.reflect.GeneratedConstructorAccessor43.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.neo4j.kernel.internal.DatabaseHealth.assertHealthy(DatabaseHealth.java:62) at org.neo4j.kernel.impl.api.Kernel.newTransaction(Kernel.java:90) at org.neo4j.kernel.impl.factory.ClassicCoreSPI.beginTransaction(ClassicCoreSPI.java:174) ... 11 more Caused by: org.neo4j.kernel.impl.store.UnderlyingStorageException: Error performing check point at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.constructCombinedFailure(CheckPointScheduler.java:100) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:81) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.neo4j.helpers.NamedThreadFactory$2.run(NamedThreadFactory.java:109) Suppressed: org.neo4j.kernel.impl.store.UnderlyingStorageException: Failed to flush at org.neo4j.kernel.impl.store.NeoStores.flush(NeoStores.java:253) at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageEngine.flushAndForce(RecordStorageEngine.java:447) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.doCheckPoint(CheckPointerImpl.java:160) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointerImpl.checkPointIfNeeded(CheckPointerImpl.java:134) at org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointScheduler$1.run(CheckPointScheduler.java:64) ... 8 more Caused by: java.io.FileNotFoundException: /home/xiujiezhang/download/neo4j-community-3.2.11/data/databases/graph.db/neostore.counts.db.b (打开的文件过多) at java.io.RandomAccessFile.open0(Native Method) at java.io.RandomAccessFile.open(RandomAccessFile.java:316) at java.io.RandomAccessFile.(RandomAccessFile.java:243)
at org.neo4j.io.fs.FileUtils.truncateFile(FileUtils.java:219)
at org.neo4j.io.fs.DefaultFileSystemAbstraction.truncate(DefaultFileSystemAbstraction.java:210)
at org.neo4j.kernel.impl.store.kvstore.KeyValueStoreFileFormat.create(KeyValueStoreFileFormat.java:150)
at org.neo4j.kernel.impl.store.kvstore.KeyValueStoreFileFormat.createStore(KeyValueStoreFileFormat.java:59)
at org.neo4j.kernel.impl.store.kvstore.RotationStrategy.next(RotationStrategy.java:106)
at org.neo4j.kernel.impl.store.kvstore.RotationState$Rotation.rotate(RotationState.java:92)
at org.neo4j.kernel.impl.store.kvstore.RotationState$Rotation.rotate(RotationState.java:52)
at org.neo4j.kernel.impl.store.kvstore.AbstractKeyValueStore$RotationTask.rotate(AbstractKeyValueStore.java:316)
at org.neo4j.kernel.impl.store.kvstore.AbstractKeyValueStore$RotationTask.rotate(AbstractKeyValueStore.java:293)
at org.neo4j.kernel.impl.store.counts.CountsTracker.rotate(CountsTracker.java:126)
at org.neo4j.kernel.impl.store.NeoStores.flush(NeoStores.java:247)
... 12 more