NethermindEth / nethermind

A robust execution client for Ethereum node operators.
https://nethermind.io/nethermind-client
GNU General Public License v3.0
1.26k stars 433 forks source link

error when closing ("Attempted to read or write protected memory") #1214

Closed one-three-three-seven closed 4 years ago

one-three-three-seven commented 4 years ago

nethermind-windows-amd64-1.4.1-9ff2311

The error is not included in the log files, only shown on console. Doesn't always happen.

2020-01-01 12:27:03.9623|Stopping blockchain processor...| 2020-01-01 12:27:03.9623|Stopping discovery app...| 2020-01-01 12:27:03.9623|Blockchain Processor shutdown complete.. please wait for all components to close| 2020-01-01 12:27:03.9753|Stopping discovery udp channel| 2020-01-01 12:27:03.9753|Discovery shutdown complete.. please wait for all components to close| 2020-01-01 12:27:05.9398|Discovered a new block 9195573 11:26:50 (0x391134...e5a827), tx count: 15 , sent by 39.97.186.72:33028| 2020-01-01 12:27:08.9647|Local peer shutdown complete.. please wait for all components to close| 2020-01-01 12:27:08.9647|Closing DBs...| Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. at RocksDbSharp.Native.rocksdb_get(IntPtr, IntPtr, Byte[], Int64, IntPtr ByRef, RocksDbSharp.ColumnFamilyHandle) at RocksDbSharp.Native.rocksdb_get(IntPtr, IntPtr, Byte[], Int64, RocksDbSharp.ColumnFamilyHandle) at Nethermind.Db.DbOnTheRocks.get_Item(Byte[]) at Nethermind.Store.StateDb.Get(Byte[]) at Nethermind.Blockchain.Synchronization.SyncServer.GetNodeData(System.Collections.Generic.IList1<Nethermind.Core.Crypto.Keccak>) at Nethermind.Network.P2P.Subprotocols.Eth.V63.Eth63ProtocolHandler.Handle(Nethermind.Network.P2P.Subprotocols.Eth.V63.GetNodeDataMessage) at Nethermind.Network.P2P.Subprotocols.Eth.V63.Eth63ProtocolHandler.HandleMessage(Nethermind.Network.Rlpx.ZeroPacket) at Nethermind.Network.P2P.Session.ReceiveMessage(Nethermind.Network.Rlpx.ZeroPacket) at Nethermind.Network.P2P.ZeroNettyP2PHandler.ChannelRead0(DotNetty.Transport.Channels.IChannelHandlerContext, Nethermind.Network.Rlpx.ZeroPacket) at DotNetty.Transport.Channels.SimpleChannelInboundHandler1[[System.__Canon, System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ChannelRead(DotNetty.Transport.Channels.IChannelHandlerContext, System.Object) at DotNetty.Transport.Channels.AbstractChannelHandlerContext.InvokeChannelRead(System.Object) at DotNetty.Common.Concurrency.AbstractEventExecutor.SafeExecute(DotNetty.Common.Concurrency.IRunnable) at DotNetty.Common.Concurrency.SingleThreadEventExecutor.RunAllTasks(DotNetty.Common.PreciseTimeSpan) at DotNetty.Common.Concurrency.SingleThreadEventExecutor.<Loop>b__26_0() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task+<>c.<.cctor>b__274_0(System.Object) at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) at System.Threading.Tasks.Task.ExecuteEntry() at System.Threading.Tasks.TaskScheduler.TryExecuteTask(System.Threading.Tasks.Task) at DotNetty.Common.Concurrency.ExecutorTaskScheduler.QueueTask(System.Threading.Tasks.Task) at System.Threading.Tasks.TaskScheduler.InternalQueueTask(System.Threading.Tasks.Task) at System.Threading.Tasks.Task.ScheduleAndStart(Boolean) at System.Threading.Tasks.TaskFactory.StartNew(System.Action, System.Threading.CancellationToken, System.Threading.Tasks.TaskCreationOptions, System.Threading.Tasks.TaskScheduler) at DotNetty.Common.Concurrency.SingleThreadEventExecutor.Loop() at DotNetty.Common.Concurrency.XThread+<>c__DisplayClass10_0.<.ctor>b__0(System.Object) at DotNetty.Common.Concurrency.XThread+<>c__DisplayClass13_0.<CreateLongRunningTask>b__0() at System.Threading.Tasks.Task.InnerInvoke() at System.Threading.Tasks.Task+<>c.<.cctor>b__274_0(System.Object) at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread) at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread) at System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object) at System.Threading.ThreadHelper.ThreadStart_Context(System.Object) at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object) at System.Threading.ThreadHelper.ThreadStart(System.Object)

tkstanczak commented 4 years ago

seen this, probably mild race condition where db is already closed but some remaining processes still try to read from rocks

tkstanczak commented 4 years ago

@one-three-three-seven I believe this is fixed now, have you seen it lately?

one-three-three-seven commented 4 years ago

No problems yet. I will just reopen if something shows up. :)