search

pterodactyl / panel

Pterodactyl® is a free, open-source game server management panel built with PHP, React, and Go. Designed with security in mind, Pterodactyl runs all game servers in isolated Docker containers while exposing a beautiful and intuitive UI to end users.
https://pterodactyl.io
Other
6.7k stars 1.7k forks source link

Docker stats not showing correctly on most recent build #4022

Closed FrankSealover closed 2 years ago

FrankSealover commented 2 years ago

Current Behavior

Currently, as of Docker version 20.10.13, build a224086, Pterodactyl does not correctly show the statistics of the container.

Docker returns proto: wrong wireType = 0 for field Hugetlb

The workaround fix is to downgrade docker or enable systemd.unified_cgroup_hierarchy=1 via grub.

Both of these work arounds are simply unacceptable and shouldn't be required to reboot to fix an issue.

Expected Behavior

Docker to correctly display container usages

Steps to Reproduce

just install pterodactyl on anything remotely new, you'll see

Panel Version

1.4.0

Wings Version

1.4.2

Games and/or Eggs Affected

No response

Docker Image

No response

Error Logs

the issue is not with the panel. it is with wings.

https://ptero.co/ufubafypuz

Is there an existing issue for this?

Jcodeerd commented 2 years ago

If the fix is unacceptable, how would you solve it than??

Software-Noob commented 2 years ago

This is not a Pterodactyl issue. As you said, downgrading or using cgroup V2 are the only workarounds until it's fixed upstream. Ref https://github.com/containerd/containerd/issues/6700 / https://github.com/moby/moby/issues/43387

matthewpi commented 2 years ago

https://github.com/moby/moby/issues/43387

It seems the issue is somewhere between Docker and containerd, not Pterodactyl. Running docker stats outputs 0 for me on a RHEL 8 machine, which tells me that Ptero is not the culprit.

The workaround for now seems to be to downgrade containerd.io, not switch to cgroups v1 or downgrade Docker itself.

If you believe the workarounds are "simply unacceptable", then why don't you spend a bit of time searching issues or finding what the actual cause of the problem is rather than expecting everyone else to do it.

FrankSealover commented 2 years ago

moby/moby#43387

It seems the issue is somewhere between Docker and containerd, not Pterodactyl. Running docker stats outputs 0 for me on a RHEL 8 machine, which tells me that Ptero is not the culprit.

The workaround for now seems to be to downgrade containerd.io, not switch to cgroups v1 or downgrade Docker itself.

If you believe the workarounds are "simply unacceptable", then why don't you spend a bit of time searching issues or finding what the actual cause of the problem is rather than expecting everyone else to do it.

That doesn't make sense. Downgrading containerd and docker may be a work around along with what is said within the discord, but no other containers have the issues even on my RHEL systems. The only close consistenty is the UUID naming scheme of Pterodactyl containers 


ead6188138dd   8bccc268-d1e2-445c-a2a1-e01e4384a60a   0.00%     0B / 0B               0.00%     0B / 0B         0B / 0B     0
150963536d9c   webmail                                0.26%     2.004GiB / 63.79GiB   3.14%     0B / 0B         0B / 0B     97
624505a04ca3   69489d86-8faf-4e23-a1d8-d13401fbd35d   0.00%     0B / 0B               0.00%     0B / 0B         0B / 0B     0
de456c3e00f5   wordpress_wordpress_1                  0.00%     20.06MiB / 63.79GiB   0.03%     39.1kB / 0B     0B / 0B     6
a36a3993ef03   nzbget                                 0.01%     6.422MiB / 63.79GiB   0.01%     39kB / 0B       0B / 0B     11
b9226ecc16e1   wikijs                                 0.01%     116MiB / 63.79GiB     0.18%     354kB / 129kB   0B / 0B     14```

Exclusively saying that it's just pterodactyl having this issue is not exactly out of the question, at least until this affects something that is not provisioned by wings (which seems to be the case addressing the consistency between the reports)
FrankSealover commented 2 years ago

If the fix is unacceptable, how would you solve it than??

I haven't I don't feel like changing cgroups in the way indicated is a good fix, that's why I recommended or mentioned to downgrade docker/containerd.io - but the point is, the issue has been in my testing only consistent for containers provisioned through wings, perhaps a hotpatch can be done to support both cgroups. But the problem is isolated to seemingly anything provisioned through wings, and unless another issue on github can be found proving this otherwise, I think its a safe assessment to say wings may need to be looked at, or at least a better explaination rather than telling users to update grub configuration files.

FrankSealover commented 2 years ago

TL:DR this issue is reproducible with wings/pterodactyl, it is not reproducable without.

matthewpi commented 2 years ago

That doesn't make sense. Downgrading containerd and docker may be a work around along with what is said within the discord, but no other containers have the issues even on my RHEL systems. The only close consistenty is the UUID naming scheme of Pterodactyl containers

...

You only need to downgrade containerd, not docker. To your statement about this only affecting Pterodactyl containers, reference the issues both SoftwareNoob and I linked above. You are trying to draw conclusions from a small dataset and lack of understanding of how our software works.

Wings interacts with Docker's APIs to manage containers, we have no direct control over how the stats are actually pulled from the system or what cgroups are used. This could be related to a setting we use when creating containers, but it works perfectly fine on containerd.io v1.4.13. I'm still going to assume that even if we can make tweaks in Wings to "fix" the problem, the actual problem lies elsewhere, even if it is an undocumented breaking change.

FrankSealover commented 2 years ago

reference the issues both SoftwareNoob and I linked above. You are trying to draw conclusions from a small dataset and lack of

I would ask if the situation is related to Pterodactyl then, and go from there. I'm going based off of the consistencies there are, and I don't think by default docker names containers based on UUID.

DaneEveritt commented 2 years ago

This is an issue with a dependency, therefore I am closing this issue.

Software-Noob commented 2 years ago

I would ask if the situation is related to Pterodactyl then, and go from there. I'm going based off of the consistencies there are, and I don't think by default docker names containers based on UUID.

@FrankSealover From further investigation on containerd, the issue occurs when oom killer is disabled, which is also a default option for Pterodactyl containers. This is probably why only Pterodactyl containers appear to have this issue for you.

They are aware of it on the containerd team

mind-overflow commented 2 years ago

Hey @Software-Noob, is there any update on this? Are the only solutions either downgrading docker or enabling OOM kiler? Can you link me to the official issue in containerd? I can't seem to find it. Thanks!

Software-Noob commented 2 years ago

You can find them linked in the responses above. The issue is also pinned on this repository with a fix released in Docker 20.10.16.

mind-overflow commented 2 years ago

Sorry, I completely missed that link. Thank you very much.