search

pmmp / PocketMine-MP

A server software for Minecraft: Bedrock Edition in PHP
https://pmmp.io
GNU Lesser General Public License v3.0
3.28k stars 1.55k forks source link

The server does not terminate forever when it crashes #5234

Closed Nerahikada closed 1 year ago

Nerahikada commented 2 years ago

Issue description

Only under certain conditions, threads do not terminate and the server does not terminate forever.

Steps to reproduce the issue

  1. Start the server with the loop flag enabled.
  2. Create and start a thread.
  3. Crashes the server.
  4. The server will not terminate and will not restart.

OS and versions

Plugins

Requires a plugin that starts a thread that extends the pocketmine\thread\Thread class.

Crashdump, backtrace or other files

$thread = new class extends Thread{
    protected function onRun() : void{
        while(!$this->isKilled){
            usleep(50);
        }
    }
};
$thread->start();

$this->getScheduler()->scheduleDelayedTask(new ClosureTask(function() : void{
    throw new \RuntimeException();
}), 20 * 120);
DaisukeDaisuke commented 2 years ago

I use the deepl translation.

It seems that pmmp does not call (ThreadManager)->stopAll() when stopped by "Forcing server shutdown". This problem is not replicated when stopping with the stop command.

code

open code(duplication) ```php isKilled){ sleep(1); var_dump($this->isKilled); } } }; $thread->start(); $this->getScheduler()->scheduleDelayedTask(new ClosureTask(function() : void{ throw new \RuntimeException(); }), 20 * 120); } } ```

console output

In the case of "Forcing server shutdown", "Stopping other threads" is not performed as compared to a normal shutdown.

Forcing server shutdown ``` [10:25:19.259] [Server thread/EMERGENCY]: Forcing server shutdown [10:25:19.260] [Server thread/DEBUG]: Disabling all plugins [10:25:19.260] [Server thread/INFO]: Disabling DevTools v1.15.0 [10:25:19.260] [Server thread/INFO]: Disabling test v0.0.1 [10:25:19.261] [Server thread/DEBUG]: Unloading all worlds [10:25:19.261] [Server thread/INFO]: Unloading world "world" [10:25:19.261] [Server thread/DEBUG]: [World: world] Cancelling unfulfilled generation requests [10:25:19.263] [Server thread/DEBUG]: Removing event handlers [10:25:19.264] [Server thread/DEBUG]: Shutting down async task worker pool [10:25:19.267] [Server thread/DEBUG]: Saving properties [10:25:19.268] [Server thread/DEBUG]: Closing console bool(false) [10:25:19.421] [Server thread/DEBUG]: Stopping network interfaces [10:25:19.421] [Server thread/DEBUG]: Stopping network interface pocketmine\network\mcpe\raklib\RakLibInterface [10:25:19.433] [RakLib thread/DEBUG]: Graceful shutdown complete [10:25:19.535] [Server thread/DEBUG]: Stopping network interface pocketmine\network\mcpe\raklib\RakLibInterface [10:25:19.539] [RakLib thread/DEBUG]: Graceful shutdown complete Killed ```
"Forcing server shutdown" within 120 seconds ``` [09:20:55.441] [Server thread/EMERGENCY]: Forcing server shutdown [09:20:55.441] [Server thread/DEBUG]: Disabling all plugins [09:20:55.441] [Server thread/INFO]: Disabling DevTools v1.15.0 [09:20:55.442] [Server thread/INFO]: Disabling test v0.0.1 [09:20:55.442] [Server thread/DEBUG]: Unloading all worlds [09:20:55.443] [Server thread/INFO]: Unloading world "world" [09:20:55.443] [Server thread/DEBUG]: [World: world] Cancelling unfulfilled generation requests [09:20:55.444] [Server thread/DEBUG]: Removing event handlers [09:20:55.445] [Server thread/DEBUG]: Shutting down async task worker pool [09:20:55.448] [Server thread/DEBUG]: Saving properties [09:20:55.449] [Server thread/DEBUG]: Closing console [09:20:55.591] [Server thread/DEBUG]: Stopping network interfaces [09:20:55.592] [Server thread/DEBUG]: Stopping network interface pocketmine\network\mcpe\raklib\RakLibInterface [09:20:55.598] [RakLib thread/DEBUG]: Graceful shutdown complete [09:20:55.700] [Server thread/DEBUG]: Stopping network interface pocketmine\network\mcpe\raklib\RakLibInterface [09:20:55.711] [RakLib thread/DEBUG]: Graceful shutdown complete --- Waiting 114 seconds to throttle automatic restart (you can kill the process safely now) --- bool(false) bool(false) bool(false) bool(false) bool(false) ^C ```
Normal shutdown ``` [09:09:50.444] [Server thread/INFO]: [CONSOLE: Stopping the server] [09:09:50.471] [Server thread/DEBUG]: Disabling all plugins [09:09:50.472] [Server thread/INFO]: Disabling DevTools v1.15.0 [09:09:50.472] [Server thread/INFO]: Disabling test v0.0.1 [09:09:50.473] [Server thread/DEBUG]: Unloading all worlds [09:09:50.473] [Server thread/INFO]: Unloading world "world" [09:09:50.474] [Server thread/DEBUG]: [World: world] Cancelling unfulfilled generation requests [09:09:50.475] [Server thread/DEBUG]: Removing event handlers [09:09:50.475] [Server thread/DEBUG]: Shutting down async task worker pool [09:09:50.481] [Server thread/DEBUG]: Saving properties [09:09:50.482] [Server thread/DEBUG]: Closing console [09:09:50.647] [Server thread/DEBUG]: Stopping network interfaces [09:09:50.647] [Server thread/DEBUG]: Stopping network interface pocketmine\network\mcpe\raklib\RakLibInterface [09:09:50.656] [RakLib thread/DEBUG]: Graceful shutdown complete [09:09:50.790] [Server thread/DEBUG]: Stopping network interface pocketmine\network\mcpe\raklib\RakLibInterface [09:09:50.799] [RakLib thread/DEBUG]: Graceful shutdown complete [09:09:50.928] [Server thread/INFO]: Stopping other threads [09:09:50.941] [Server thread/DEBUG]: Stopping Thread@anonymous/home/daiuskedaisuke/pmmp/PocketMine-MP/plugins/test/src/Main.php:13$27d thread [09:09:50.945] [Server thread/DEBUG]: Thread@anonymous/home/daiuskedaisuke/pmmp/PocketMine-MP/plugins/test/src/Main.php:13$27d thread stopped successfully. [09:09:50.945] [Server thread/DEBUG]: Stopping Server Killer thread [09:09:50.949] [Server thread/DEBUG]: Server Killer thread stopped successfully. ```

Other information

This problem does not occur with \Thread by itself.(without pmmp)

open code and console output ```php isKilled){ usleep(100000); var_dump($this->isKilled); } } }; $thread->start(); usleep(200000); $thread->isKilled = true; $thread->join(); ``` ``` bool(false) bool(true) ```

https://github.com/pmmp/PocketMine-MP/blob/237c2866e0d2988857a8f6423cae4e8291832099/src/Server.php#L1675-L1682 https://github.com/pmmp/PocketMine-MP/blob/0234d821f5a9bba38d65fafd8fede54742dc8e4a/src/PocketMine.php#L306-L315

dktapps commented 2 years ago

I'm not able to reproduce this using WSL2.

Nerahikada commented 2 years ago

I'm not able to reproduce this using WSL2.

Did you start the server with the loop flag enabled?

dktapps commented 2 years ago

I'm not able to reproduce this using WSL2.

Did you start the server with the loop flag enabled?

Yes. The script isn't able to influence anything within PM, so it doesn't make any difference.

Nerahikada commented 2 years ago

When the server is started with loop enabled, the group process ID of the PHP process will be that of bash. Therefore, the server will not terminate because it is trying to kill a group process ID that does not exist.

$ pstree -acutspng
init,1,
  ├─{init},6,
  ├─init,7,7
  │   └─init,8,7
  │       └─bash,9,9,user
  │           └─bash,6432,6432 ./start.sh -l
  │               └─php,6435,6432
  │                   ├─{php},6488,6432
  │                   ├─{php},6489,6432
  │                   ├─{php},6537,6432
  │                   ├─{php},6538,6432
  │                   ├─{php},6539,6432
  │                   ├─{php},6540,6432
  │                   └─php,6541,6432                                                                                                                   ...
  └─init,222,222
      └─init,223,222
          └─bash,224,224,user
              └─pstree,6543,6543 -acutspng
dktapps commented 2 years ago

As I said, I tested this myself, it doesn't reproduce. This must be specific to whatever distro you're using.

nasiridrishi commented 2 years ago

I have had the same issue on ubuntu but same does not seem to exist on Mac

xerenahmed commented 2 years ago

https://github.com/pmmp/PocketMine-MP/blob/bd13f39156dfe56cc99c6e733a34f2304832c174/src/utils/Process.php#L128

https://github.com/pmmp/PocketMine-MP/blob/bd13f39156dfe56cc99c6e733a34f2304832c174/src/utils/Process.php#L137-L142

When subprocess is set to true, server not killing actually with right PID

dktapps commented 2 years ago

When the server is started with loop enabled, the group process ID of the PHP process will be that of bash. Therefore, the server will not terminate because it is trying to kill a group process ID that does not exist.

$ pstree -acutspng
init,1,
  ├─{init},6,
  ├─init,7,7
  │   └─init,8,7
  │       └─bash,9,9,user
  │           └─bash,6432,6432 ./start.sh -l
  │               └─php,6435,6432
  │                   ├─{php},6488,6432
  │                   ├─{php},6489,6432
  │                   ├─{php},6537,6432
  │                   ├─{php},6538,6432
  │                   ├─{php},6539,6432
  │                   ├─{php},6540,6432
  │                   └─php,6541,6432                                                                                                                   ...
  └─init,222,222
      └─init,223,222
          └─bash,224,224,user
              └─pstree,6543,6543 -acutspng

It does exist. I tested it on WSL2 and got the exact same behaviour, and it terminates itself just fine.

This must be some specific behaviour to the distro or version thereof you're using.

dktapps commented 2 years ago

Reproduced in Ubuntu 20.04.4.

dktapps commented 2 years ago

Autopsy: The server assumes that it has its own process group with GID = -PID. However, this isn't the case when the server process is directly invoked from bash - it belongs to the same process group as bash.

Using the actual GID via getmygid() is also not an option, because that would also terminate the bash process responsible for the loop.

dktapps commented 1 year ago

Fixed by 9b43ddecb.