500 Internal Server Error

[MAXOUXAX]

Background:

• Panel or Daemon: Panel & Wings
• Version of Panel/Daemon: Panel@1.0.1 - Wings@1.0.0
• Server's OS: Debian 10
• Your Computer's OS & Browser: Brave (chromium based browser)

Describe the bug This bug did not occur on the latest release candidate, but I noticed that my status page, which periodically checks the status of the servers using the Client API, was recording some failures after updating my panel and wings.

So as soon as I received the failure notification, I rushed to the panel to see what was going on, and I saw an internal server 500 error in the console following a request to the /api/client/servers/{id}/resources endpoint: Failed to load the resource: the server responded with a status of 500 (Internal Server Error).

I changed the disk_check_interval field in the configuration because checking disk usage takes about 5 minutes, and the default delay is close to that, so it was far from ideal in my configuration, so I set it to 7200.

I think the 500 error occurs when the disk usage check is in progress (given its recurrence)

Panel: php -v » PHP 7.4.8 Daemon: uname -a and docker info » Linux vps-ee197cf8 4.19.0-10-cloud-amd64 #1 SMP Debian 4.19.132-1 (2020-07-24) x86_64 GNU/Linux

To Reproduce Steps to reproduce the behavior:

1. Edit the disk_check_interval field to an higher value
2. Visit the panel while the check is in progress (or use a tool to periodically check the endpoint of the api like Statping)
3. Open the browser console
4. See error

Expected behavior Same behavior before version 1.0.0, no error 500 when the disk usage check is running.

[notAreYouScared]

Could you get us the error log from the panel? They're located in /var/www/pterodactyl/storage/log

[MAXOUXAX]

This error is logged every minute. I'm not sure if it's related to the 500 on my status page, but it's the only error in the logs, there's nothing else.

production.ERROR: Pterodactyl\Exceptions\Model\DataValidationException: Data integrity exception encountered while performing database write operation. {"name":["The name field is required."]} in /var/www/pterodactyl/app/Models/Model.php:60
Stack trace:
#0 /var/www/pterodactyl/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(381): Pterodactyl\Models\Model::Pterodactyl\Models\{closure}()
#1 /var/www/pterodactyl/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(226): Illuminate\Events\Dispatcher->Illuminate\Events\{closure}()
#2 /var/www/pterodactyl/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(199): Illuminate\Events\Dispatcher->dispatch()

[DaneEveritt]

Please provide logs from the time when you're seeing these errors. The error you provided doesn't appear to have anything to do with the endpoint you're reporting issues with.

[MAXOUXAX]

Ah, I didn't make myself understood, but the error I sent is the only one I have in the logs, I don't have any other error, even when my status page gets a 500 error. The file I'm checking is laravel-2020-10-17.log, is this the correct log file ?

[DaneEveritt]

You might need to check the webserver error logs as well, and wings.

[MAXOUXAX]

Nothing in the web server logs, where are Wings' logs ?

[DaneEveritt]

/var/log/pterodactyl

But if that too has no logs then we'll have to close this issue for now because there is nothing here that I can really use to reproduce this issue, and a 500 error should absolutely have been logged somewhere lining up with the time in which you saw the error.

[MAXOUXAX]

I looked at all the logs (web server, php, wings, pterodactyl.app-error, etc) and I didn't find any other error than the one I sent. I also checked the access.log nginx file, and, surprisingly, there is no log for this endpoint in the given time period, which is quite odd.

My status page checks the status of the server every 5 minutes, and this happens as you can see on the screen about 5 times a day.

I understand that, being a 500 error, something should be logged, but I literally checked all the log files I could, and found nothing. Would running wings in debug mode help in any way?

[DaneEveritt]

Can you show the response body of the failing requests? You can try debug mode on wings, but a 500 error from wings would have been logged anyways.

[MAXOUXAX]

I'm afraid that's not possible, Statping doesn't display any additional information in the logs. I will try to get the error in my browser to get more information.

[DaneEveritt]

Okay, I'm closing this issue for the time being until enough information can be gathered that there is an actionable goal. 👍

[MAXOUXAX]

Here's the response I got from the request:

{
    "errors": [
        {
            "code": "UnexpectedValueException",
            "status": "500",
            "detail": "An unexpected error was encountered while processing this request, please try again."
        }
    ]
}

I checked again all the log files I could find, and nothing was modified at the time of the error, nothing in the nginx logs, nothing in the php logs, nothing in the wings logs, absolutely nothing... I'm starting to think that error logging has been disabled somewhere, so the error is "silent" and there's simply nothing written in the logs.

You can see on the screenshot that the request failed once with the 500 error, and then the request worked fine, and at the time of the request the disk usage check was in progress (see the graphs on my netdata dashboard)

Again, the problem began to occur after I updated to 1.0.0 from 1.0.0-rc.6.

[MAXOUXAX]

I've made updates to the latest versions (Panel@1.1.1 & Wings@1.1.1) and the problem is still there. On the home page of the panel, I saw my 4 servers, 3 of which were online and the one where the disk usage check was in progress was displayed as offline with a loading animation on the resource usage overview.

I finally managed to get some logs! @DaneEveritt

[DaneEveritt]

[2020-11-11 17:00:38] production.WARNING: GuzzleHttp\Exception\ConnectException: cURL error 28: Operation timed out after 5000 milliseconds with 0 bytes received (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) in /var/www/pterodactyl/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:200

I suppose it might be a possibility that the API endpoint for returning server details hangs when the disk check is running? That is my best guess, can you confirm that seems to line up with what you're seeing?

[MAXOUXAX]

Yeah that's what I thought too ^^

I think the 500 error occurs when the disk usage check is in progress (given its recurrence)

[MAXOUXAX]

Also got another error on the same line of code but with another server.

[2020-11-18 13:51:53] production.WARNING: GuzzleHttp\Exception\ConnectException: cURL error 28: Resolving timed out after 3000 milliseconds (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) in /var/www/pterodactyl/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:200
[...]
13 /var/www/pterodactyl/vendor/guzzlehttp/guzzle/src/Client.php(95): GuzzleHttp\Client->request('get', '/api/servers/e0...', Array)

Here are the full logs (the stack trace is identical)

[DaneEveritt]

That error looks like DNS resolution issues, not anything to do with Pterodactyl specifically.

[pythoniaweb]

This issue looks like code issue i got this too and cant even fix it support in discord dont helped

[G-Basil]

For me the resolve for this was php artisan cache:clear

[Dampfwalze]

My issue was, that the panel could not verify the SSL-certificate of the wing. I explaned it here: https://github.com/pterodactyl/panel/issues/4862#issuecomment-1879754219