SEServerExtender / EssentialsPlugin

Dedicated Essentials Plugin
GNU Lesser General Public License v3.0
18 stars 18 forks source link

[CRASH]Server fails to restart, server crashes/hangs, random high pings #63

Open FoxWyvern opened 9 years ago

FoxWyvern commented 9 years ago

This is probably not an Essentials problem but I'd at least like to open it as a topic of discussion so that more can be learned about the issue. I'm going to attempt to post this as a bug on Keen's forum as well as I do believe this is more of an issue with the new version of space engineers although I'm posting it here in case it has something to do with SESE having not had an update for the newest version of SE.

I've noticed the following issues since the last Space Engineers update

-Server crashes roughly every other day. -Random players get extremely high ping but doesn't affect anyone else connected to the server at the same time. -Server does not automatically restart. (Essentials feature)

Server Currently Using: Space Engineers: 01.089.014 SESE: 1.88.11.543 Essentials: 1.13.2.2443

Not using: wvm, wrr, voxel concealment, or GetOutOfMySandbox

I am not noticing anything of any value in any of the logs, which I'll post below. When the server crashes, the server initiates a shut down and unloads the essentials plugin (which is probably why the server does not automatically restart). It then hangs and still allows incoming connections but anyone who trys to connect will get stuck at the "Downloading World" screen.

My server is setup as an instance and has restart in the target line with the timed restart and unresponsive restart enabled for essentials. However, the actual service for the instance has been set to manual instead of automatic which I do not believe is an issue in this case but mentioning it just in case.

All crashes to my knowledge have occurred when a player is mining, although one player said he "was about to start mining." However, none of the other server admins I spoke to have noticed any abnormal behavior with their own servers.

Additionally, random players at random times are experiencing excessively high pings, in the 10s or even 100s or thousands high before dropping back down to high 1,000s and randomly jumping up and down till that player reconnects. These ping spikes appear to happen randomly and to random players and do not affect other players who happen to be on the server at the same time.

None of these issues appear to be able to be repeatable on demand so easily identifying a script or plugin that may or may not be causing this is extremely difficult. However, both problems have appeared to crop up at the same time.

---Logs---

End of World Log 2015-07-08 12:45:15.953 - Thread: 5 -> Saving world - START 2015-07-08 12:45:15.953 - Thread: 5 -> Making world state snapshot. 2015-07-08 12:45:15.953 - Thread: 5 -> Before snapshot.: GC Memory: 906,958,432 B 2015-07-08 12:45:16.254 - Thread: 5 -> After snapshot.: GC Memory: 934,467,440 B 2015-07-08 12:45:16.254 - Thread: 5 -> Directory cleanup: GC Memory: 934,467,440 B 2015-07-08 12:45:16.254 - Thread: 5 -> Saving world - END 2015-07-08 12:45:16.255 - Thread: 16 -> Session snapshot save - START 2015-07-08 12:45:16.255 - Thread: 16 -> Checking file access for files in target dir. 2015-07-08 12:45:19.975 - Thread: 16 -> Session snapshot save - END 2015-07-08 12:45:21.000 - Thread: 5 -> GC Memory: 944,627,336 B 2015-07-08 12:45:51.009 - Thread: 5 -> GC Memory: 989,936,952 B 2015-07-08 12:46:21.014 - Thread: 5 -> GC Memory: 869,994,408 B 2015-07-08 12:46:51.029 - Thread: 5 -> GC Memory: 886,281,824 B 2015-07-08 12:47:21.043 - Thread: 5 -> GC Memory: 897,212,888 B 2015-07-08 12:47:28.515 - Thread: 5 -> MyGlobalEvent.RecalculateActivationTime: 2015-07-08 12:47:28.515 - Thread: 5 -> Next activation in 00:12:33.7776015 2015-07-08 12:47:51.049 - Thread: 5 -> GC Memory: 910,033,776 B 2015-07-08 12:48:21.055 - Thread: 5 -> GC Memory: 858,803,848 B 2015-07-08 12:48:51.064 - Thread: 5 -> GC Memory: 858,481,064 B 2015-07-08 12:49:21.066 - Thread: 5 -> GC Memory: 868,297,048 B

------End of SESE Log------ 2015-07-08 12:30:16.2237|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b7|Asynchronous Save Setup Started: 299.0171ms 2015-07-08 12:30:19.9579|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b5|Asynchronous Save Completed: 4033.2307ms 2015-07-08 12:35:16.2288|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b7|Asynchronous Save Setup Started: 299.0171ms 2015-07-08 12:35:19.8720|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b5|Asynchronous Save Completed: 3942.2255ms 2015-07-08 12:40:16.2250|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b7|Asynchronous Save Setup Started: 292.0167ms 2015-07-08 12:40:19.8472|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b5|Asynchronous Save Completed: 3914.2239ms 2015-07-08 12:45:16.2541|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b7|Asynchronous Save Setup Started: 317.0181ms 2015-07-08 12:45:19.9764|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b5|Asynchronous Save Completed: 4039.231ms 2015-07-08 12:49:25.8944|Info|SEModAPIExtensions.API.Server.RunServer|Server has stopped running 2015-07-08 12:49:25.8994|Info|SEModAPIExtensions.API.PluginManager.UnloadPlugin|Unloading plugin 'f767bda5-0a8c-4813-85fd-a5a2341778e8'

-----End of Plugin Log----- 2015-07-08 12:44:59.6252|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:05.2265|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:16.8282|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:17.2282|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:21.2304|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:28.4318|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:31.2320|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:32.4321|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:34.8372|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:45:57.6465|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:46:01.6467|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:46:06.4470|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:46:12.0473|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:46:16.0476|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:46:18.0487|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:47:19.6542|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:49:13.2687|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:49:21.2702|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-08 12:49:25.8994|Info|EssentialsPlugin.Essentials.Shutdown|Shutting down plugin: Dedicated Server Essentials - 1.13.2.2443

Note: Problem getting entities, skipping update seems to be a common error flood when the server UPS drops dramatically due to players mining.

FoxWyvern commented 9 years ago

Noticed this in the server prompt window just now...

2015-07-08 17:34:46.9284|Error|ProcessCommunication: Problem getting entities, s kipping update 2015-07-08 17:34:52.5287|Error|ProcessCommunication: Problem getting entities, s kipping update 2015-07-08 17:34:54.5298|Error|ProcessCommunication: Problem getting entities, s kipping update 2015-07-08 17:35:05.3494|Error|Error getting entity list, skipping check |System.InvalidOperationException: Collection was modified; enumeration oper ation may not execute. at System.Collections.Generic.HashSet1.Enumerator.MoveNext() at Sandbox.ModAPI.MyEntitiesHelper_ModAPI.Sandbox.ModAPI.IMyEntities.GetEntit ies(HashSet1 entities, Func`2 collect) at EssentialsPlugin.EntityManagers.EntityManagement.CheckAndConcealEntities()

2015-07-08 17:35:09.3347|Error|ProcessCommunication: Problem getting entities, s kipping update 2015-07-08 17:35:15.7400|Error|ProcessCommunication: Problem getting entities, s kipping update 2015-07-08 17:35:17.7401|Error|ProcessCommunication: Problem getting entities, s kipping update

FoxWyvern commented 9 years ago

Another crash, this time with a more detailed error report:

plugins log

2015-07-10 12:48:32.6437|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-10 12:48:33.0437|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-10 12:48:35.0448|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-10 12:48:37.4449|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-10 12:48:41.4472|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-10 12:48:45.0474|Error|EssentialsPlugin.ProcessHandlers.ProcessCommunication.Handle|ProcessCommunication: Problem getting entities, skipping update 2015-07-10 12:48:52.4178|Info|EssentialsPlugin.Essentials.Shutdown|Shutting down plugin: Dedicated Server Essentials - 1.13.2.2443 2015-07-10 12:48:52.4358|Error|EssentialsPlugin.ProcessHandlers.ProcessGreeting.Handle|System.Threading.ThreadAbortException: Thread was being aborted. at SEModAPIInternal.API.Common.SandboxGameAssemblyWrapper.GameAction(Action action) at EssentialsPlugin.ProcessHandlers.ProcessGreeting.Handle() 2015-07-10 12:48:52.4358|Warn|EssentialsPlugin.Essentials+<>cDisplayClass3.b2|Handler Problems: 1000 - System.Threading.ThreadAbortException: Thread was being aborted. at EssentialsPlugin.ProcessHandlers.ProcessGreeting.Handle() at EssentialsPlugin.Essentials.<>cDisplayClass3.b2()

SESE Log:

2015-07-10 12:45:01.8856|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b7|Asynchronous Save Setup Started: 265.0152ms 2015-07-10 12:45:05.6628|Info|SEModAPIInternal.API.Common.WorldManager+<>cDisplayClass8.b5|Asynchronous Save Completed: 4042.2312ms 2015-07-10 12:48:52.4178|Info|SEModAPIExtensions.API.Server.RunServer|Server has stopped running 2015-07-10 12:48:52.4178|Info|SEModAPIExtensions.API.PluginManager.UnloadPlugin|Unloading plugin 'f767bda5-0a8c-4813-85fd-a5a2341778e8' 2015-07-10 12:48:52.4358|Error|SEModAPIInternal.API.Common.SandboxGameAssemblyWrapper.GameAction|System.Threading.ThreadAbortException: Thread was being aborted. at System.Threading.WaitHandle.WaitOneNative(SafeHandle waitableSafeHandle, UInt32 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext) at System.Threading.WaitHandle.InternalWaitOne(SafeHandle waitableSafeHandle, Int64 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext) at SEModAPIInternal.API.Common.SandboxGameAssemblyWrapper.GameAction(Action action)

tdmike commented 9 years ago

Is it a creative server ?