pterodactyl / panel

Pterodactyl® is a free, open-source game server management panel built with PHP, React, and Go. Designed with security in mind, Pterodactyl runs all game servers in isolated Docker containers while exposing a beautiful and intuitive UI to end users.
https://pterodactyl.io
Other
6.69k stars 1.7k forks source link

Restarting all server at node after unexpected EOF #4033

Closed deff-dev closed 2 years ago

deff-dev commented 2 years ago

Current Behavior

At random time wings restart all servers on node. identical problem: https://discord.com/channels/122900397965705216/658550982975946762/942932339296706580

P.S the error appeared after updating the wings to the latest version (1.6.1)

Expected Behavior

Servers not restart if have warn:

 INFO: [Apr  3 03:16:12.019] detected server as entering a crashed state; running crash handler server=a3b12602-238b-4c0e-9280-67cc85013836
error processing scanner line in console output container_id=id error=unexpected EOF

 WARN: [Apr  3 03:16:12.053] error processing scanner line in console output container_id=a3b12602-238b-4c0e-9280-67cc85013836 error=unexpected EOF

Stacktrace:
unexpected EOF
github.com/pterodactyl/wings/environment/docker.(*Environment).scanOutput
        /home/runner/work/wings/wings/environment/docker/container.go:351
runtime.goexit
        /opt/hostedtoolcache/go/1.17.6/x64/src/runtime/asm_amd64.s:1581

Also i noticed when i get this error, wings automaticly restarted Выделение_040

Steps to Reproduce

I think error with decode lines at console, but i check server console logs, and didn't see problems with decode line Выделение_041

Panel Version

1.7.0

Wings Version

1.6.1

Games and/or Eggs Affected

No response

Docker Image

No response

Error Logs

https://pastebin.com/raw/UpzxeYAV

Is there an existing issue for this?

parkervcp commented 2 years ago

Can you get actual loge from around that error/crash? So far you've given us very little.

All I see from this is that Wings saw your server enter a crashed state and forced a restart.

deff-dev commented 2 years ago

here is the whole log during the crash: https://pastebin.com/raw/7Fneq3X6 no more info at logs, also i can enable debug mode at config.

deff-dev commented 2 years ago

logs with wings crash at another my node: https://pastebin.com/raw/sRZsnWeN image

deff-dev commented 2 years ago

All I see from this is that Wings saw your server enter a crashed state and forced a restart.

The problem is different, if crash wings, all servers at node restarted

DaneEveritt commented 2 years ago

Wings does not restart all servers when it crashes. Also that is a handled error, it just warns and I believe we stop parsing server console messages for a second or two to try reconnecting.

If Wings is being restarted at the same time I'd be more inclined to believe you have some different issue at play causing Docker to restart, and that just happens to be the last thing Wings processes.

deff-dev commented 2 years ago

If Wings is being restarted at the same time I'd be more inclined to believe you have some different issue at play causing Docker to restart, and that just happens to be the last thing Wings processes.

Yes, you are right, docker crash, maybe there logs will give you more information: https://pastebin.com/raw/wdr1LUve

DaneEveritt commented 2 years ago
Apr  3 03:16:06 Euclid kernel: [3550950.920894] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/docker.service,task=dockerd,pid=545072,uid=0
Apr  3 03:16:06 Euclid kernel: [3550950.920950] Out of memory: Killed process 545072 (dockerd) total-vm:36510808kB, anon-rss:20697636kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:45728kB oom_score_adj:-500
Apr  3 03:16:06 Euclid kernel: [3550951.323578] oom_reaper: reaped process 545072 (dockerd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Apr  3 03:16:18 Euclid systemd[1]: docker.service: Main process exited, code=killed, status=9/KILL

Not a bug with Pterodactyl.

deff-dev commented 2 years ago

Yeap, i found mistake, thx.