Rocologo / MobHunting

Mob hunting plugin for Minecraft Servers
24 stars 22 forks source link

The server won't stop #747

Closed Innesity closed 2 years ago

Innesity commented 2 years ago

Hi. The server will stuck on disabling mobhunting and never stop itself after you will kill the server on pterodactyl. Can you please fix this? Screenshot_19

Rocologo commented 2 years ago

This is probably because another database operation happened earlier in the server log. I need to see this error?

Innesity commented 2 years ago

This is probably because another database operation happened earlier in the server log. I need to see this error?

[deleted link] -- this is the full log. The server will never stop by itself, you'll have to force stop it or kill the container.

Innesity commented 2 years ago

@Rocologo Hey. Can you take a look, please?

Rocologo commented 2 years ago

I'm on vacation without my computer. You should look for SQL errors in the server log, to find the root cause for why the plugin can't stop.

Sendt fra Samsung Galaxy S10+

-------- Oprindelig besked -------- Fra: Innesity @.> Dato: 24.07.2022 13.08 (GMT+03:00) Til: Rocologo/MobHunting @.> Cc: Rocologo @.>, Mention @.> Emne: Re: [Rocologo/MobHunting] The server won't stop (Issue #747)

@Rocologohttps://github.com/Rocologo Hey. Can you take a look, please?

— Reply to this email directly, view it on GitHubhttps://github.com/Rocologo/MobHunting/issues/747#issuecomment-1193286269, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAHWEBPU4X26DA2BJ6EHFIDVVUIX5ANCNFSM53MRZIBQ. You are receiving this because you were mentioned.Message ID: @.***>

AlohaLegend commented 2 years ago

Hey there wanted to jump in, I am having the same issue ever since updating to 1.19. Sometimes It will disable Mobhunt later on in the shutdown process. Worst case scenarios Mobhunt will be the first plugin to be disabled meaning all plugins beyond it will not be shutdown incorrectly. This has happened only a few times but has reset entire plugins. My log looks essentially the same as Innesity: Ending in [MobHunting] Disabling MobHunting v8.1.5

I have searched for sql errors and as of now have found none. The server will not restart properly when being online for quite a few hours. If the server has only been online for a couple hours it will reset normally.

Thanks in advance!

Rocologo commented 2 years ago

Sorry for the big delay. I have not had time to look at it because of vacation.

I have checked the full log. I need you to enable debug so I can get more information

/mh debug

and the I would like to see a new log?

Rocologo commented 2 years ago

The end of the log is fine, for now.

Innesity commented 2 years ago

Sorry for the big delay. I have not had time to look at it because of vacation.

I have checked the full log. I need you to enable debug so I can get more information

/mh debug

and the I would like to see a new log?

I will try later tonight and come back with the log to you :)

Innesity commented 2 years ago

Started debugging, will provide the full log later this evening, when the Cloud System will restart all subservers

Innesity commented 2 years ago

Sorry for the big delay. I have not had time to look at it because of vacation.

I have checked the full log. I need you to enable debug so I can get more information

/mh debug

and the I would like to see a new log?

[deleted link] -- here is the new log

Rocologo commented 2 years ago

Thank you. This line conform that it is because of MobHunting.

  | [03:00:35] [Server thread/INFO]: [MobHunting] [Debug] waitForEmptyQueue: Waiting for 607+0 tasks to finish before closing connections.

It tells me that MobHunting cant save to the database. 607 updates is waiting before MobHunting will continue. This usually happens when you see an SQL error in the log. In your case I dont see any SQL errors so maybe MobHunting dont have permission to write to the Database? Are you using MySQL og Sqlite ? If you use MySQL you should try to re-grant permissions .

1) are you using MySQL or Sqlite ? 1.5) If you are using MySQL check the SQL permissions 1.6) If you are using Sqlite, try to stop the server. Rename MobHuning.db to MobHunting.db.bak and start the server. This will regenerate the database and database tables but also reset player stats! This will tell you if there is a problem with the database. 2) Please show me the "beginning" of the server log, so I can check what MubHunting is doing and chack that it seems to be okay.

Innesity commented 2 years ago

Thank you. This line conform that it is because of MobHunting.

  | [03:00:35] [Server thread/INFO]: [MobHunting] [Debug] waitForEmptyQueue: Waiting for 607+0 tasks to finish before closing connections. It tells me that MobHunting cant save to the database. 607 updates is waiting before MobHunting will continue. This usually happens when you see an SQL error in the log. In your case I dont see any SQL errors so maybe MobHunting dont have permission to write to the Database? Are you using MySQL og Sqlite ? If you use MySQL you should try to re-grant permissions .

  1. are you using MySQL or Sqlite ? 1.5) If you are using MySQL check the SQL permissions 1.6) If you are using Sqlite, try to stop the server. Rename MobHuning.db to MobHunting.db.bak and start the server. This will regenerate the database and database tables but also reset player stats! This will tell you if there is a problem with the database.
  2. Please show me the "beginning" of the server log, so I can check what MubHunting is doing and chack that it seems to be okay.

The log is very big because of debugging, so I uploaded on my OneDrive account. Here is the link: [deleted link]

Note that I will delete the log after 6 hours because of security issues that may appear by sharing IPs and other things. I am using SQLite. I wanted to move to MySQL, but I don't know if there's a way to migrate data without any loss.

Rocologo commented 2 years ago

Unfortunately you cant migrate data :-( but Sqlite should work anyway. Make sure that MobHuting.db is writeable and not write protectet or something.

You can send the beginning of the log to my personal email if that better. Its

rocologo AT hotmail.com

Innesity commented 2 years ago

Unfortunately you cant migrate data :-( but Sqlite should work anyway. Make sure that MobHuting.db is writeable and not write protectet or something.

You can send the beginning of the log to my personal email if that better. Its

rocologo AT hotmail.com

I sent the email.

Rocologo commented 2 years ago

I have actually found a MobHunting error in the log file you send me. This is probably the reason for the problem. I cant tell why you get at ConcurrentModification on your server and I dont get it on my server, but now I have something to look at when I get home from work.

[15:11:26] [BGC StoreThread/WARN]: Exception in thread "BGC StoreThread" java.util.ConcurrentModificationException [15:11:26] [BGC StoreThread/WARN]: at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1597) [15:11:26] [BGC StoreThread/WARN]: at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1630) [15:11:26] [BGC StoreThread/WARN]: at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1628) [15:11:26] [BGC StoreThread/WARN]: at MobHunting-8.1.5.jar//one.lindegaard.Core.rewards.RewardBlockManager.save(RewardBlockManager.java:83) [15:11:26] [BGC StoreThread/WARN]: at MobHunting-8.1.5.jar//one.lindegaard.Core.storage.DataStoreManager$StoreThread.run(DataStoreManager.java:236)

Innesity commented 2 years ago

I have actually found a MobHunting error in the log file you send me. This is probably the reason for the problem. I cant tell why you get at ConcurrentModification on your server and I dont get it on my server, but now I have something to look at when I get home from work.

[15:11:26] [BGC StoreThread/WARN]: Exception in thread "BGC StoreThread" java.util.ConcurrentModificationException [15:11:26] [BGC StoreThread/WARN]: at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1597) [15:11:26] [BGC StoreThread/WARN]: at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1630) [15:11:26] [BGC StoreThread/WARN]: at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1628) [15:11:26] [BGC StoreThread/WARN]: at MobHunting-8.1.5.jar//one.lindegaard.Core.rewards.RewardBlockManager.save(RewardBlockManager.java:83) [15:11:26] [BGC StoreThread/WARN]: at MobHunting-8.1.5.jar//one.lindegaard.Core.storage.DataStoreManager$StoreThread.run(DataStoreManager.java:236)

Okay. Take your time man, and thanks for your help!

Rocologo commented 2 years ago

Your welcome. It was exactly errors like this I was looking for. Its always a problem to find error which I dont see on my own Two servers and this one I have never seen, but I think I know how to solve it.

Rocologo commented 2 years ago

Please test this snapshot and see if the ConcurrentModificationException disapears. And if you still have problems with closing the server.

https://jenkins.lindegaard.one/job/MobHunting/121/

Let me know the result :-)

Search your log for ConcurrentModificationException :-)

NeverGrowUp commented 2 years ago

Can confirm that im getting a similar issue as above on our production server. Although we never see the "ConcurrentModificationException" error. But i notice that the rewards.yml in BoG does not update periodically as defined by save_period: 6000 in bagofgoldcore.yml

Testing locally with the above snapshot, i can confirm the save period now works. Although testing locally i've never had the problem of it just hanging on shutdown. (but does happen on live server everytime, and need to soft kill the session.)

Will endeavour to test on our live server tomorrow.

Rocologo commented 2 years ago

Hmm. I hope I have not introduced a new bug 😞

NeverGrowUp commented 2 years ago

Sorry. My burst above may have been misleading. The updated build 121 you linked above seems to have fixed the issue. (for us) Tested on our live server and all seems to be going ok, and getting successful shutdowns. Also now seeing the periodic saves of [BagOfGoldCore][MobHunting] [Debug] Saved 966 rewards to disk as defined by save_period: 6000 in bagofgoldcore.yml.

So, so far so good!! Thx

Rocologo commented 2 years ago

Perfect. I will see if I can make a release tonight or tomorrow so this bug can be fixed for others too. Thank you for your help.

Rocologo commented 2 years ago

Version 8.1.7 released with this fix

AlohaLegend commented 1 year ago

Hello, popping into this closed thread. I had this issue during this period - See above. Everything has worked since this release. Then yesterday after making no chances to mobhunt it started happening again. Now my server will not shut down correctly. I see there is an error with my customitemslib. I will be deleting the customitemslib config folder to see if a fresh install config might resolve the situation(I have done this before). I'm currently running the latest paper build 1.19.3. If you need anything else from me let me know. Thanks in advance.

Rocologo commented 1 year ago

I need to see the error you talk about?

In most cases this type of error cased bu an SQL error. Please check your server log for errors and show me the error

AlohaLegend commented 1 year ago

Update on this - I was going to run a mh debug restart but it would seem that deleting the customitemslib folder resolved the issue. Will update if it happens again.