Closed ReVoLt112 closed 8 months ago
Yep, same on my side. It seems, that the scheduled task is still runnning, but I also get an HTTP 500 on the frontend.
Same here, only started the container yesterday (sat) around 15.00 (UTC+1), last run was at 23.00 (UTC+1), this morning I also saw the error 500.
Something happening on sat/sun night?
@michaelmittermair can you also confirm similar behavior?
Yes, look similar. I checked my container. It was running longer than just 1 day, but the last run was yesterday (SAT) 23:47pm (UTC +1).
Same here, no logs that give any useful information. Last successful run was (SAT) 23:42 (UTC+1). Next run would have been at (SUN) 00:12 (UTC+1). So it appears that something happened around midnight.
Same here
same here. logs don't indicate anything has gone wrong. simply after about a week, server crashes, and web ui only loads showing 500 | SERVER ERROR
@te5s3rakt could you do one additional check (as I believe this could narrow down the probleem a bit). Could you check if this happens after midnight on Saturdays?
@te5s3rakt could you do one additional check (as I believe this could narrow down the probleem a bit). Could you check if this happens after midnight on Saturdays?
sure thing. my local time?
Yes, what UTC offset are you in?
Yes, what UTC offset are you in?
UTC + 8
I've just restarted and updated my container. I'll check next weekend how the container is going.
I had this too this morning. A restart of my docker container restored access for me.
I noticed in the logs that Laravel was throwing an error saying that no application encryption key has been specified.
same here
I had this too this morning. A restart of my docker container restored access for me.
I noticed in the logs that Laravel was throwing an error saying that no application encryption key has been specified.
I'm seeing this same error in the logs [2024-01-23 16:30:35] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)
i dont think that the app key is the reason for the server crash. I have a key generated and dont have a warning on start.
My guess is that there is something wrong in: /app/Console/Kernel.php
/**
* Perform system maintenance weekly on Sunday morning,
* start off the week nice and fresh.
*/
$schedule->command(SystemMaintenance::class)->weeklyOn(0)
->timezone($settings->timezone ?? 'UTC');
I have also a 500 on one of my instances, got the following error:
192.168.176.23 - - [24/Jan/2024:11:17:44 +0100] "GET /api/speedtest/latest HTTP/1.1" 200 386 "-" "-" 192.168.176.23 - - [24/Jan/2024:11:17:45 +0100] "GET /api/speedtest/latest HTTP/1.1" 200 386 "-" "-" [2024-01-24 10:17:58] production.ERROR: Unsupported cipher or incorrect key length. Supported ciphers are: aes-128-cbc, aes-256-cbc, aes-128-gcm, aes-256-gcm. {"exception":"[object] (RuntimeException(code: 0): Unsupported cipher or incorrect key length. Supported ciphers are: aes-128-cbc, aes-256-cbc, aes-128-gcm, aes-256-gcm. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/Encrypter.php:55) [stacktrace]
"}
My instance (Docker, using Postgres) just stopped responding at 00:00 EST (05:00 GMT) on Sunday morning. The same thing happened last Sunday. Logs seem to indicate some kind of issue connecting to Postgres. However, the Postgres server is running fine and connected to other apps. A restart of the Docker container fixes the issue.
The logs show the following:
[2024-01-28 05:00:42] production.ERROR: SQLSTATE[HY000] [2002] Connection refused (Connection: mysql, SQL: select * from `results` order by `created_at` desc limit 1) {"exception":"[object] (Illuminate\\Database\\QueryException(code: 2002): SQLSTATE[HY000] [2002] Connection refused (Connection: mysql, SQL: select * from `results` order by `created_at` desc limit 1) at /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php:822)
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(976): Illuminate\\Database\\Connection->runQueryCallback()
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(955): Illuminate\\Database\\Connection->tryAgainIfCausedByLostConnection()
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(778): Illuminate\\Database\\Connection->handleQueryException()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(407): Illuminate\\Database\\Connection->run()
#4 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2751): Illuminate\\Database\\Connection->select()
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2740): Illuminate\\Database\\Query\\Builder->runSelect()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(3294): Illuminate\\Database\\Query\\Builder->Illuminate\\Database\\Query\\{closure}()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2739): Illuminate\\Database\\Query\\Builder->onceWithColumns()
#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(737): Illuminate\\Database\\Query\\Builder->get()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(721): Illuminate\\Database\\Eloquent\\Builder->getModels()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/BuildsQueries.php(333): Illuminate\\Database\\Eloquent\\Builder->get()
#11 /var/www/html/app/Http/Controllers/API/Speedtest/GetLatestController.php(18): Illuminate\\Database\\Eloquent\\Builder->first()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\\Http\\Controllers\\API\\Speedtest\\GetLatestController->__invoke()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(43): Illuminate\\Routing\\Controller->callAction()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(259): Illuminate\\Routing\\ControllerDispatcher->dispatch()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(205): Illuminate\\Routing\\Route->runController()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(806): Illuminate\\Routing\\Route->run()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(159): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(125): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequest()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(87): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequestUsingNamedLimiter()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\ThrottleRequests->handle()
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\\Pipeline\\Pipeline->then()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\\Routing\\Router->runRouteWithinStack()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\\Routing\\Router->runRoute()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\\Routing\\Router->dispatchToRoute()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\\Routing\\Router->dispatch()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#31 /var/www/html/vendor/livewire/livewire/src/Features/SupportDisablingBackButtonCache/DisableBackButtonCacheMiddleware.php(19): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Livewire\\Features\\SupportDisablingBackButtonCache\\DisableBackButtonCacheMiddleware->handle()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#41 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#43 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(62): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
Yes, what UTC offset are you in?
UTC + 8
I've just restarted and updated my container. I'll check next weekend how the container is going.
Was right on time. Sunday morning container sh!t itself.
I get the same error after midnight Sunday, 500 | SERVER ERROR
Same again for me this Sunday. Error once again this one (when browsing to the server):
production.ERROR: No application encryption key has been specified
same problem here, running on RPi 4 with Postgres DB. I can see these two error messages if that helps.
2024-01-28T07:58:15.918491042Z [previous exception] [object] (PDOException(code: 2002): SQLSTATE[HY000] [2002] Connection refused at /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php:65)
2024-01-28T07:58:22.570684825Z [2024-01-28 07:58:22] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)
I have an app key specified, if I recreate the container it will work for a few days.
restarting the container produces this error:
2024-01-28T08:02:58.099306835Z SQLSTATE[HY000] [2002] Connection refused (Connection: mysql, SQL: delete f
2024-01-28T08:02:58.099321409Z rom cache
)
2024-01-28T08:02:58.099335575Z
2024-01-28T08:02:58.099385074Z
2024-01-28T08:02:58.099653069Z In Connector.php line 65:
2024-01-28T08:02:58.099683272Z
2024-01-28T08:02:58.099833751Z SQLSTATE[HY000] [2002] Connection refused
2024-01-28T08:02:58.099928101Z
2024-01-28T08:02:58.099948231Z
2024-01-28T08:02:58.131371371Z s6-rc: warning: unable to start service laravel-automations: command exited 1
2024-01-28T08:02:58.136766293Z /run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
2024-01-28T08:02:58.136880161Z prog: fatal: stopping the container.
I get the same error after midnight Sunday, 500 | SERVER ERROR
I think mine also stops working every Sunday.
Same here.
My logs:
`[2024-01-28 08:21:08] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79) [stacktrace]
"} 192.168.25.7 - - [28/Jan/2024:08:21:08 +0000] "GET / HTTP/1.1" 500 6628 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 OPR/106.0.0.0" [2024-01-28 08:21:08] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79) [stacktrace]
"} `
yep i get this sunday morning problem. restarting the speedtest and db containers gets it running again.
yep i get this sunday morning problem. restarting the speedtest and db containers gets it running again.
I only restart my speedtest container. my db one never presents any issues with anything else I have connected to it.
same here (stopps with Error 500) Database is mariadb - restarting web container fixes the issue
yep i get this sunday morning problem. restarting the speedtest and db containers gets it running again.
I only restart my speedtest container. my db one never presents any issues with anything else I have connected to it.
i first tried restarting the speedtest container, but no joy. when i restarted both it worked again.
same here (stopps with Error 500) Database is mariadb - restarting web container fixes the issue
Same here. First week with this running, discovered it stopped last night. Restart of the Speedtest container fixed it.
As a temporary workaround, you can add the following to your /etc/crontab on the docker host system to restart the container every sunday morning at 00:02 local time:
2 0 * * 0 root docker restart speedtest-tracker >/dev/null 2>&1
i dont think that the app key is the reason for the server crash. I have a key generated and dont have a warning on start. My guess is that there is something wrong in: /app/Console/Kernel.php /* Perform system maintenance weekly on Sunday morning, start off the week nice and fresh. / $schedule->command(SystemMaintenance::class)->weeklyOn(0) ->timezone($settings->timezone ?? 'UTC');
I agree this is probably the source of the issue. SystemMaintenance is sourced from app/console/commands/SystemMaintenance.php
. I suspect the issue must be in this function:
/**
* Clear and refresh the cache.
*/
protected function refreshCache(): void
{
Artisan::call('optimize:clear');
Artisan::call('optimize');
}
I can confirm that I can replicate the crash at a specific time by editing /app/console/Kernel.php and then restarting the container:
$schedule->command(SystemMaintenance::class)->weeklyOn(0,'12:12')
This causes a crash at 12:12 local time.
If I comment out both calls to Artisan in refreshCache () (SystemMaintenance.php), the server stays up even through a scheduled maintenance. Leaving either one uncommented will still result in a crash. My guess is that the optimize command somehow clears out the app key, even for people who have specified one, leading to a crash.
@mmomjian nice observation. Is the key stored in the db? Could you check if it is indeed removed after the optimize/optimize:clear is called?
Just adding 'me too'. I see this issue happen roughly once a week, with scheduled checks set to ever half an hour.
[2024-01-28 10:23:56] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\\Encryption\\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)
[stacktrace]
@nwagenmakers I don't see any reference to the key in the postgres database, but I'm not sure exactly where to look.
same here (stopps with Error 500) Database is mariadb - restarting web container fixes the issue
Same here. First week with this running, discovered it stopped last night. Restart of the Speedtest container fixed it.
MariaDB here as well. So probably explains why we both can just refresh the speedtest container to get it working, whereas others need to refresh both speedtest and their db containers.
@nwagenmakers I don't see any reference to the key in the postgres database, but I'm not sure exactly where to look.
I crawled mine twice and couldn't find it either.
@te5s3rakt I run Postgres and it’s never required a restart, so I’m not sure if that’s related.
I have also a 500 on one of my instances, got the following error:
Unsupported cipher or incorrect key length. Supported ciphers are: aes-128-cbc, aes-256-cbc, aes-128-gcm, aes-256-gcm.
I have the same exact error regardless of whether I'm accessing the service via HTTP or HTTPS.
@te5s3rakt I run Postgres and it’s never required a restart, so I’m not sure if that’s related.
I run Postgres as well and face a 500 error when trying to access the service via the browser directly, again regardless whether HTTP or HTTPS is used for browsing the service. Interestingly though, it worked earlier this day for a brief moment where I could see stats on the page, but since then I get the error quoted above.
And more surprisingly, the integration into homepage shows an active entry for the service which even updates slightly.
The problem therefore seems to only affect browser access (tested with Firefox 122.0 and Edge 121.0). The API seems to be working though.
The reported error stems from a TLS handshake failure where the target service called doesn't support the ciphers the client supports. aes-128-cbc
indicates a TLS 1.2 secured endpoint. TLS 1.3 i.e. has dropped support of AES_CBC ciphers as they aren't up to modern security standards further. However, AES_GCM ciphers should be usable by TLS 1.3 AFAIK. As this error occurs when accessing the service either via HTTP or HTTPS I guess the cause of this error might be within the service itself. But I haven't looked at the internals of the service yet.
Getting this error too, later version + maria db + API_KEY set as ENV.
@mmomjian nice find. I've tried to run the maintenance commands manually, and seems like it didn't crash the UI. root@02cacc2411dd:/var/www/html# php artisan optimize:clear
INFO Clearing cached bootstrap files.
events .................................................................................................................................. 2ms DONE views ................................................................................................................................... 6ms DONE cache ................................................................................................................................... 2ms DONE route ................................................................................................................................... 1ms DONE config .................................................................................................................................. 1ms DONE compiled ................................................................................................................................ 1ms DONE
root@02cacc2411dd:/var/www/html# php artisan optimize
INFO Caching the framework bootstrap files.
config ................................................................................................................................. 20ms DONE routes ................................................................................................................................. 21ms DONE
I didn't have time to change the schedule (no editor in the docker :) ), but it should be pretty much the same.
Just to add my 'me too'. Speedtest front end is always visible. Speed tests stop working reliably each Sunday at 0000. The last test is always posted late Saturday night.
For those having an issue, follow the debug steps in this comment on #1035 https://github.com/alexjustesen/speedtest-tracker/issues/1035#issuecomment-1924907035
@alexjustesen here are my results. The cache driver is set to file. I installed using the docker compose instructions at the website. Are we supposed to use the .env.example file from the github repo for the /config folder? My .env file is autogenerated as the instructions don't actually specify to create a .env file.
# php artisan about
Environment ................................................................
Application Name ......................................... Speedtest Tracker
Laravel Version .................................................... 10.40.0
PHP Version ......................................................... 8.2.14
Composer Version ..................................................... 2.6.6
Environment ..................................................... production
Debug Mode ............................................................. OFF
URL .............................................................. localhost
Maintenance Mode ....................................................... OFF
Cache ......................................................................
Config .............................................................. CACHED
Events .......................................................... NOT CACHED
Routes .............................................................. CACHED
Views ............................................................... CACHED
Drivers ....................................................................
Broadcasting ........................................................... log
Cache ................................................................. file
Database ............................................................. pgsql
Logs ................................................................ stderr
Mail .................................................................. smtp
Queue ............................................................. database
Session ........................................................... database
Filament ...................................................................
Packages ................... filament, forms, notifications, support, tables
Version ............................................................ v3.1.47
Views ........................................................ NOT PUBLISHED
Livewire ...................................................................
Livewire ............................................................ v3.3.5
Speedtest Tracker ..........................................................
Out of date ............................................................ Yes
Version ............................................................ v0.14.4
/config/.env
:
# A full list of options can be found in the '.env.example' file.
# Application key
# Used for encryption where needed, if a key is not generated run: php artisan key:generate
APP_KEY=base64:<redacted>
# Application URL
# Not used in the URL but effects links in emails and notifications: https://docs.speedtest-tracker.dev/faqs#links-in-emails-dont-point-to-the-correct-url
APP_URL=http://localhost
# SMTP mail config
# Only SMTP protocol is supported at this time: https://docs.speedtest-tracker.dev/settings/notifications/mail
MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="Speedtest Tracker"
# Telegram bot config
TELEGRAM_BOT_TOKEN=null
@mmomjian correct, when you start up the container if a .env
file doesn't exist it's copied from .env.production
.
If you change anything in this file make sure to restart your container so the changes are picked up. I can see your cache driver is set to file
, because we don't mount the storage
directory with the correct permissions you'll want to use the database
cache driver.
@alexjustesen I am still on v0.14.4, I believe the bug you had linked to may be a different issue, sorry for the confusion on my end. The issue in this thread is a crash every Sunday morning at 00:00 local time. Do you want me to try setting the cache driver to database and see if it stays up this week? What is the correct environment variable to set to change the cache driver?
@alexjustesen I am still on v0.14.4, I believe the bug you had linked to may be a different issue, sorry for the confusion on my end. The issue in this thread is a crash every Sunday morning at 00:00 local time. Do you want me to try setting the cache driver to database and see if it stays up this week? What is the correct environment variable to set to change the cache driver?
Please do, it should be set to database
.
I added to CACHE_DRIVER: 'database'
to the docker compose file and I now get:
Drivers ..........................................................................................................................................
Broadcasting ................................................................................................................................. log
Cache ................................................................................................................................... database
I'll report back on Sunday.
For those having an issue, follow the debug steps in this comment on #1035 #1035 (comment)
Environment ...................................................................................................................................... Application Name ............................................................................................................... Speedtest Tracker Laravel Version .......................................................................................................................... 10.40.0 PHP Version ............................................................................................................................... 8.2.14 Composer Version ........................................................................................................................... 2.6.6 Environment ........................................................................................................................... production Debug Mode ................................................................................................................................... OFF URL ............................................................................................................................. spt.i.home.org Maintenance Mode ............................................................................................................................. OFF
Cache ............................................................................................................................................ Config .................................................................................................................................... CACHED Events ................................................................................................................................ NOT CACHED Routes .................................................................................................................................... CACHED Views ..................................................................................................................................... CACHED
Drivers .......................................................................................................................................... Broadcasting ................................................................................................................................. log Cache ....................................................................................................................................... file Database ................................................................................................................................... mysql Logs ...................................................................................................................................... stderr Mail ........................................................................................................................................ smtp Queue ................................................................................................................................... database Session ................................................................................................................................. database
Filament ......................................................................................................................................... Packages ......................................................................................... filament, forms, notifications, support, tables Version .................................................................................................................................. v3.1.47 Views .............................................................................................................................. NOT PUBLISHED
Livewire ......................................................................................................................................... Livewire .................................................................................................................................. v3.3.5
Speedtest Tracker ................................................................................................................................ Out of date .................................................................................................................................. Yes Version .................................................................................................................................. v0.14.4
Docker-compose speedtest: container_name: speedtest image: ghcr.io/alexjustesen/speedtest-tracker:latest volumes:
spt.i.home.org
)Documentation say the default file cache is probably best in most cases, why database then ?
stopping the container and restarting fixes the issue
I can confirm this is happening every Sunday - probably exactly at 0:00 - restarting the container fixes this as reported. It just happened again - I've now set up a cron job at 0:01 every Sunday that restarts the container as a workaround.
Environment ......................................................................................................................................
Application Name ............................................................................................................... Speedtest Tracker
Laravel Version .......................................................................................................................... 10.43.0
PHP Version ............................................................................................................................... 8.2.15
Composer Version ........................................................................................................................... 2.6.6
Environment ........................................................................................................................... production
Debug Mode ................................................................................................................................... OFF
URL .................................................................................................................................... localhost
Maintenance Mode ............................................................................................................................. OFF
Cache ............................................................................................................................................
Config .................................................................................................................................... CACHED
Events ................................................................................................................................ NOT CACHED
Routes .................................................................................................................................... CACHED
Views ..................................................................................................................................... CACHED
Drivers ..........................................................................................................................................
Broadcasting ................................................................................................................................. log
Cache ................................................................................................................................... database
Database ................................................................................................................................... mysql
Logs ...................................................................................................................................... stderr
Mail ........................................................................................................................................ smtp
Queue ................................................................................................................................... database
Session ................................................................................................................................. database
Filament .........................................................................................................................................
Packages ......................................................................................... filament, forms, notifications, support, tables
Version .................................................................................................................................. v3.2.23
Views .............................................................................................................................. NOT PUBLISHED
Livewire .........................................................................................................................................
Livewire .................................................................................................................................. v3.4.4
Speedtest Tracker ................................................................................................................................
Out of date ................................................................................................................................... No
Version .................................................................................................................................. v0.14.6
The exact moment the error occurs the webpage will only display error 500 and the following appears in the log continuously until the container is restartet: (offset from 0:00 by 1 hour due to timezones)
[2024-02-03 23:00:12] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\\Encryption\\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/helpers.php(307): Illuminate\\Encryption\\EncryptionServiceProvider->Illuminate\\Encryption\\{closure}()
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(77): tap()
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(60): Illuminate\\Encryption\\EncryptionServiceProvider->key()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(32): Illuminate\\Encryption\\EncryptionServiceProvider->parseKey()
#4 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(908): Illuminate\\Encryption\\EncryptionServiceProvider->Illuminate\\Encryption\\{closure}()
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\\Container\\Container->build()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\\Container\\Container->resolve()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\\Foundation\\Application->resolve()
#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\\Container\\Container->make()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(1066): Illuminate\\Foundation\\Application->make()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(982): Illuminate\\Container\\Container->resolveClass()
#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(943): Illuminate\\Container\\Container->resolveDependencies()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\\Container\\Container->build()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\\Container\\Container->resolve()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\\Foundation\\Application->resolve()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\\Container\\Container->make()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(172): Illuminate\\Foundation\\Application->make()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\\Pipeline\\Pipeline->then()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\\Routing\\Router->runRouteWithinStack()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\\Routing\\Router->runRoute()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\\Routing\\Router->dispatchToRoute()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\\Routing\\Router->dispatch()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#24 /var/www/html/vendor/livewire/livewire/src/Features/SupportDisablingBackButtonCache/DisableBackButtonCacheMiddleware.php(19): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Livewire\\Features\\SupportDisablingBackButtonCache\\DisableBackButtonCacheMiddleware->handle()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(36): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\HandleCors->handle()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\TrustProxies->handle()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(175): Illuminate\\Pipeline\\Pipeline->then()
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(144): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter()
#41 /var/www/html/public/index.php(51): Illuminate\\Foundation\\Http\\Kernel->handle()
#42 {main}
"}
It seams every Sunday at 0:00 the app looeses it's app key - even through it is set by the environment variable in docker and recognized in the log on startup:
🔒 SSL_MODE has been set to MIXED, setting the web server to work in HTTP + HTTPS...
🐇 Configuring Speedtest Tracker...
🔒 Fixing app path file permissions...
✅ Permissions fixed.
✅ Environment file exists.
🔗 Creating symlinks for config and log files...
✅ Symlinks created.
✅ An application key environment variable exists.
💰 Building the cache...
✅ Cache set.
🚛 Migrating the database...
✅ Database migrated.
✅ All set, Speedtest Tracker started.
[03-Feb-2024 22:35:59] NOTICE: fpm is running, pid 117
[03-Feb-2024 22:35:59] NOTICE: ready to handle connections
[03-Feb-2024 22:35:59] NOTICE: systemd monitor interval set to 10000ms
As the log shows I restartet the container about 30 minutes before midnight (due to updating the image to the latest version). Exactly at midnight it errored out - less than an hour after starting it. So the amount the time the container was running doesn't seem to matter at all - it will stop working at Sunday 0:00 regardless of how long it was running before (mere minutes or a week).
For me, midnight on Sunday was just over an hour ago as of this post, and sure enough, I couldn't access the container. It was working fine earlier today (Saturday). I hasn't noticed this before, but I was checking the container's stability after I'd figured out the issue referenced in #1087. I simply restarted the container and all is well again.
There's definitely something still wrong here.
I’m EST and it was midnight 39 minutes ago and that’s when it stopped working for me again too. A reboot fixes it.
Died at 12:00am local time. It seems that the cache was reset to file
. A restart of the Docker container fixes the issue, and it now reports Cache...database
again.
This output is at the time of the crash, before container restart.
# php artisan about
Environment .....................................................................................................
Application Name .............................................................................. Speedtest Tracker
Laravel Version ......................................................................................... 10.40.0
PHP Version .............................................................................................. 8.2.14
Composer Version .......................................................................................... 2.6.6
Environment .......................................................................................... production
Debug Mode .................................................................................................. OFF
URL ................................................................................................... localhost
Maintenance Mode ............................................................................................ OFF
Cache ...........................................................................................................
Config ................................................................................................... CACHED
Events ............................................................................................... NOT CACHED
Routes ................................................................................................... CACHED
Views .................................................................................................... CACHED
Drivers .........................................................................................................
Broadcasting ................................................................................................ log
Cache ...................................................................................................... file
Database .................................................................................................. mysql
Logs ..................................................................................................... stderr
Mail ....................................................................................................... smtp
Queue .................................................................................................. database
Session ................................................................................................ database
Filament ........................................................................................................
Packages ........................................................ filament, forms, notifications, support, tables
Version ................................................................................................. v3.1.47
Views ............................................................................................. NOT PUBLISHED
Livewire ........................................................................................................
Livewire ................................................................................................. v3.3.5
Speedtest Tracker ...............................................................................................
Out of date ................................................................................................. Yes
Version ................................................................................................. v0.14.4
Hi, after one week my speedtest-tracker stops working. Sometimes it just doesnt execute scheduled tests, sometimes i get a HTTP500. Has someone also noticed same behaviour?
Unfortunately no helpful logs are produced... I am watching 3 instances and all 3 crash nearly same moment from saturday to sunday