search

alexjustesen / speedtest-tracker

Speedtest Tracker is a self-hosted internet performance tracking application that runs speedtest checks against Ookla's Speedtest service.
https://speedtest-tracker.dev/
MIT License
2.8k stars 107 forks source link

Stops working after a week #1065

Closed ReVoLt112 closed 8 months ago

ReVoLt112 commented 9 months ago

Hi, after one week my speedtest-tracker stops working. Sometimes it just doesnt execute scheduled tests, sometimes i get a HTTP500. Has someone also noticed same behaviour?

Unfortunately no helpful logs are produced... I am watching 3 instances and all 3 crash nearly same moment from saturday to sunday

michaelmittermair commented 9 months ago

Yep, same on my side. It seems, that the scheduled task is still runnning, but I also get an HTTP 500 on the frontend.

nwagenmakers commented 9 months ago

Same here, only started the container yesterday (sat) around 15.00 (UTC+1), last run was at 23.00 (UTC+1), this morning I also saw the error 500.

Something happening on sat/sun night?

@michaelmittermair can you also confirm similar behavior?

michaelmittermair commented 9 months ago

Yes, look similar. I checked my container. It was running longer than just 1 day, but the last run was yesterday (SAT) 23:47pm (UTC +1).

bive242 commented 9 months ago

Same here, no logs that give any useful information. Last successful run was (SAT) 23:42 (UTC+1). Next run would have been at (SUN) 00:12 (UTC+1). So it appears that something happened around midnight.

TempuzFugit commented 9 months ago

Same here

te5s3rakt commented 9 months ago

same here. logs don't indicate anything has gone wrong. simply after about a week, server crashes, and web ui only loads showing 500 | SERVER ERROR

nwagenmakers commented 9 months ago

@te5s3rakt could you do one additional check (as I believe this could narrow down the probleem a bit). Could you check if this happens after midnight on Saturdays?

te5s3rakt commented 9 months ago

@te5s3rakt could you do one additional check (as I believe this could narrow down the probleem a bit). Could you check if this happens after midnight on Saturdays?

sure thing. my local time?

nwagenmakers commented 9 months ago

Yes, what UTC offset are you in?

te5s3rakt commented 9 months ago

Yes, what UTC offset are you in?

UTC + 8

I've just restarted and updated my container. I'll check next weekend how the container is going.

mattjgalloway commented 9 months ago

I had this too this morning. A restart of my docker container restored access for me.

I noticed in the logs that Laravel was throwing an error saying that no application encryption key has been specified.

coro1404 commented 9 months ago

same here

ThomasHFWright commented 9 months ago

I had this too this morning. A restart of my docker container restored access for me.

I noticed in the logs that Laravel was throwing an error saying that no application encryption key has been specified.

I'm seeing this same error in the logs [2024-01-23 16:30:35] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)

alexjustesen commented 9 months ago

For the app key error: https://docs.speedtest-tracker.dev/faqs#i-get-a-warning-on-container-start-up-that-the-app_key-is-missing

ReVoLt112 commented 9 months ago

i dont think that the app key is the reason for the server crash. I have a key generated and dont have a warning on start.

My guess is that there is something wrong in: /app/Console/Kernel.php

     /**
      * Perform system maintenance weekly on Sunday morning,
      * start off the week nice and fresh.
      */
     $schedule->command(SystemMaintenance::class)->weeklyOn(0)
         ->timezone($settings->timezone ?? 'UTC');
Leave-MeAlone commented 9 months ago

I have also a 500 on one of my instances, got the following error:

192.168.176.23 - - [24/Jan/2024:11:17:44 +0100] "GET /api/speedtest/latest HTTP/1.1" 200 386 "-" "-" 192.168.176.23 - - [24/Jan/2024:11:17:45 +0100] "GET /api/speedtest/latest HTTP/1.1" 200 386 "-" "-" [2024-01-24 10:17:58] production.ERROR: Unsupported cipher or incorrect key length. Supported ciphers are: aes-128-cbc, aes-256-cbc, aes-128-gcm, aes-256-gcm. {"exception":"[object] (RuntimeException(code: 0): Unsupported cipher or incorrect key length. Supported ciphers are: aes-128-cbc, aes-256-cbc, aes-128-gcm, aes-256-gcm. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/Encrypter.php:55) [stacktrace]

0 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(32): Illuminate\Encryption\Encrypter->__construct()

1 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(908): Illuminate\Encryption\EncryptionServiceProvider->Illuminate\Encryption\{closure}()

2 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\Container\Container->build()

3 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\Container\Container->resolve()

4 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\Foundation\Application->resolve()

5 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\Container\Container->make()

6 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(1066): Illuminate\Foundation\Application->make()

7 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(982): Illuminate\Container\Container->resolveClass()

8 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(943): Illuminate\Container\Container->resolveDependencies()

9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\Container\Container->build()

10 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\Container\Container->resolve()

11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\Foundation\Application->resolve()

12 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\Container\Container->make()

13 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(172): Illuminate\Foundation\Application->make()

14 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

15 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\Pipeline\Pipeline->then()

16 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\Routing\Router->runRouteWithinStack()

17 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\Routing\Router->runRoute()

18 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\Routing\Router->dispatchToRoute()

19 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\Routing\Router->dispatch()

20 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}()

21 /var/www/html/vendor/livewire/livewire/src/Features/SupportDisablingBackButtonCache/DisableBackButtonCacheMiddleware.php(19): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

22 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Livewire\Features\SupportDisablingBackButtonCache\DisableBackButtonCacheMiddleware->handle()

23 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

24 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()

25 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull->handle()

26 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

27 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()

28 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\TrimStrings->handle()

29 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

30 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle()

31 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

32 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance->handle()

33 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(49): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

34 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Http\Middleware\HandleCors->handle()

35 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

36 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Http\Middleware\TrustProxies->handle()

37 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

38 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(175): Illuminate\Pipeline\Pipeline->then()

39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(144): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter()

40 /var/www/html/public/index.php(51): Illuminate\Foundation\Http\Kernel->handle()

41 {main}

"}

mmomjian commented 9 months ago

My instance (Docker, using Postgres) just stopped responding at 00:00 EST (05:00 GMT) on Sunday morning. The same thing happened last Sunday. Logs seem to indicate some kind of issue connecting to Postgres. However, the Postgres server is running fine and connected to other apps. A restart of the Docker container fixes the issue.

The logs show the following:

[2024-01-28 05:00:42] production.ERROR: SQLSTATE[HY000] [2002] Connection refused (Connection: mysql, SQL: select * from `results` order by `created_at` desc limit 1) {"exception":"[object] (Illuminate\\Database\\QueryException(code: 2002): SQLSTATE[HY000] [2002] Connection refused (Connection: mysql, SQL: select * from `results` order by `created_at` desc limit 1) at /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php:822)
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(976): Illuminate\\Database\\Connection->runQueryCallback()
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(955): Illuminate\\Database\\Connection->tryAgainIfCausedByLostConnection()
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(778): Illuminate\\Database\\Connection->handleQueryException()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php(407): Illuminate\\Database\\Connection->run()
#4 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2751): Illuminate\\Database\\Connection->select()
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2740): Illuminate\\Database\\Query\\Builder->runSelect()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(3294): Illuminate\\Database\\Query\\Builder->Illuminate\\Database\\Query\\{closure}()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2739): Illuminate\\Database\\Query\\Builder->onceWithColumns()
#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(737): Illuminate\\Database\\Query\\Builder->get()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php(721): Illuminate\\Database\\Eloquent\\Builder->getModels()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Concerns/BuildsQueries.php(333): Illuminate\\Database\\Eloquent\\Builder->get()
#11 /var/www/html/app/Http/Controllers/API/Speedtest/GetLatestController.php(18): Illuminate\\Database\\Eloquent\\Builder->first()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\\Http\\Controllers\\API\\Speedtest\\GetLatestController->__invoke()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(43): Illuminate\\Routing\\Controller->callAction()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(259): Illuminate\\Routing\\ControllerDispatcher->dispatch()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Route.php(205): Illuminate\\Routing\\Route->runController()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(806): Illuminate\\Routing\\Route->run()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(159): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(125): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequest()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Middleware/ThrottleRequests.php(87): Illuminate\\Routing\\Middleware\\ThrottleRequests->handleRequestUsingNamedLimiter()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Routing\\Middleware\\ThrottleRequests->handle()
#24 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\\Pipeline\\Pipeline->then()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\\Routing\\Router->runRouteWithinStack()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\\Routing\\Router->runRoute()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\\Routing\\Router->dispatchToRoute()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\\Routing\\Router->dispatch()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#31 /var/www/html/vendor/livewire/livewire/src/Features/SupportDisablingBackButtonCache/DisableBackButtonCacheMiddleware.php(19): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Livewire\\Features\\SupportDisablingBackButtonCache\\DisableBackButtonCacheMiddleware->handle()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#41 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#42 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#43 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(62): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
te5s3rakt commented 9 months ago

Yes, what UTC offset are you in?

UTC + 8

I've just restarted and updated my container. I'll check next weekend how the container is going.

Was right on time. Sunday morning container sh!t itself.

Exit24 commented 9 months ago

I get the same error after midnight Sunday, 500 | SERVER ERROR

mattjgalloway commented 9 months ago

Same again for me this Sunday. Error once again this one (when browsing to the server):

production.ERROR: No application encryption key has been specified
ShlomiD83 commented 9 months ago

same problem here, running on RPi 4 with Postgres DB. I can see these two error messages if that helps.

2024-01-28T07:58:15.918491042Z [previous exception] [object] (PDOException(code: 2002): SQLSTATE[HY000] [2002] Connection refused at /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php:65)

2024-01-28T07:58:22.570684825Z [2024-01-28 07:58:22] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)

I have an app key specified, if I recreate the container it will work for a few days.

restarting the container produces this error: 2024-01-28T08:02:58.099306835Z SQLSTATE[HY000] [2002] Connection refused (Connection: mysql, SQL: delete f
2024-01-28T08:02:58.099321409Z rom cache)
2024-01-28T08:02:58.099335575Z
2024-01-28T08:02:58.099385074Z 2024-01-28T08:02:58.099653069Z In Connector.php line 65: 2024-01-28T08:02:58.099683272Z
2024-01-28T08:02:58.099833751Z SQLSTATE[HY000] [2002] Connection refused
2024-01-28T08:02:58.099928101Z
2024-01-28T08:02:58.099948231Z 2024-01-28T08:02:58.131371371Z s6-rc: warning: unable to start service laravel-automations: command exited 1 2024-01-28T08:02:58.136766293Z /run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information. 2024-01-28T08:02:58.136880161Z prog: fatal: stopping the container.

ShlomiD83 commented 9 months ago

I get the same error after midnight Sunday, 500 | SERVER ERROR

I think mine also stops working every Sunday.

michaelmittermair commented 9 months ago

Same here.

My logs:

`[2024-01-28 08:21:08] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79) [stacktrace]

0 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/helpers.php(307): Illuminate\Encryption\EncryptionServiceProvider->Illuminate\Encryption\{closure}()

1 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(77): tap()

2 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(60): Illuminate\Encryption\EncryptionServiceProvider->key()

3 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(32): Illuminate\Encryption\EncryptionServiceProvider->parseKey()

4 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(908): Illuminate\Encryption\EncryptionServiceProvider->Illuminate\Encryption\{closure}()

5 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\Container\Container->build()

6 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\Container\Container->resolve()

7 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\Foundation\Application->resolve()

8 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\Container\Container->make()

9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(1066): Illuminate\Foundation\Application->make()

10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(982): Illuminate\Container\Container->resolveClass()

11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(943): Illuminate\Container\Container->resolveDependencies()

12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\Container\Container->build()

13 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\Container\Container->resolve()

14 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\Foundation\Application->resolve()

15 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\Container\Container->make()

16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(172): Illuminate\Foundation\Application->make()

17 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

18 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\Pipeline\Pipeline->then()

19 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\Routing\Router->runRouteWithinStack()

20 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\Routing\Router->runRoute()

21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\Routing\Router->dispatchToRoute()

22 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\Routing\Router->dispatch()

23 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\Foundation\Http\Kernel->Illuminate\Foundation\Http\{closure}()

24 /var/www/html/vendor/livewire/livewire/src/Features/SupportDisablingBackButtonCache/DisableBackButtonCacheMiddleware.php(19): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

25 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Livewire\Features\SupportDisablingBackButtonCache\DisableBackButtonCacheMiddleware->handle()

26 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

27 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(31): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()

28 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull->handle()

29 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

30 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(40): Illuminate\Foundation\Http\Middleware\TransformsRequest->handle()

31 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\TrimStrings->handle()

32 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

33 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\ValidatePostSize->handle()

34 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance->handle()

36 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(49): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

37 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Http\Middleware\HandleCors->handle()

38 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

39 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Http\Middleware\TrustProxies->handle()

40 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}()

41 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(175): Illuminate\Pipeline\Pipeline->then()

42 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(144): Illuminate\Foundation\Http\Kernel->sendRequestThroughRouter()

43 /var/www/html/public/index.php(51): Illuminate\Foundation\Http\Kernel->handle()

44 {main}

"} 192.168.25.7 - - [28/Jan/2024:08:21:08 +0000] "GET / HTTP/1.1" 500 6628 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 OPR/106.0.0.0" [2024-01-28 08:21:08] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\Encryption\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79) [stacktrace]

0 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/helpers.php(307): Illuminate\Encryption\EncryptionServiceProvider->Illuminate\Encryption\{closure}()

1 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(77): tap()

2 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(60): Illuminate\Encryption\EncryptionServiceProvider->key()

3 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(32): Illuminate\Encryption\EncryptionServiceProvider->parseKey()

4 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(908): Illuminate\Encryption\EncryptionServiceProvider->Illuminate\Encryption\{closure}()

5 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\Container\Container->build()

6 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\Container\Container->resolve()

7 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\Foundation\Application->resolve()

8 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\Container\Container->make()

9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(1066): Illuminate\Foundation\Application->make()

10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(982): Illuminate\Container\Container->resolveClass()

11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(943): Illuminate\Container\Container->resolveDependencies()

12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\Container\Container->build()

13 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\Container\Container->resolve()

14 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\Foundation\Application->resolve()

15 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\Container\Container->make()

16 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(255): Illuminate\Foundation\Application->make()

17 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(213): Illuminate\Foundation\Http\Kernel->terminateMiddleware()

18 /var/www/html/public/index.php(55): Illuminate\Foundation\Http\Kernel->terminate()

19 {main}

"} `

crispybegs commented 9 months ago

yep i get this sunday morning problem. restarting the speedtest and db containers gets it running again.

te5s3rakt commented 9 months ago

yep i get this sunday morning problem. restarting the speedtest and db containers gets it running again.

I only restart my speedtest container. my db one never presents any issues with anything else I have connected to it.

coro1404 commented 9 months ago

same here (stopps with Error 500) Database is mariadb - restarting web container fixes the issue

crispybegs commented 9 months ago

yep i get this sunday morning problem. restarting the speedtest and db containers gets it running again.

I only restart my speedtest container. my db one never presents any issues with anything else I have connected to it.

i first tried restarting the speedtest container, but no joy. when i restarted both it worked again.

Johnster001 commented 9 months ago

same here (stopps with Error 500) Database is mariadb - restarting web container fixes the issue

Same here. First week with this running, discovered it stopped last night. Restart of the Speedtest container fixed it.

mmomjian commented 9 months ago

As a temporary workaround, you can add the following to your /etc/crontab on the docker host system to restart the container every sunday morning at 00:02 local time:

2 0 * * 0 root docker restart speedtest-tracker >/dev/null 2>&1

mmomjian commented 9 months ago

i dont think that the app key is the reason for the server crash. I have a key generated and dont have a warning on start. My guess is that there is something wrong in: /app/Console/Kernel.php /* Perform system maintenance weekly on Sunday morning, start off the week nice and fresh. / $schedule->command(SystemMaintenance::class)->weeklyOn(0) ->timezone($settings->timezone ?? 'UTC');

I agree this is probably the source of the issue. SystemMaintenance is sourced from app/console/commands/SystemMaintenance.php. I suspect the issue must be in this function:

    /**
     * Clear and refresh the cache.
     */
    protected function refreshCache(): void
    {
        Artisan::call('optimize:clear');

        Artisan::call('optimize');
    }

I can confirm that I can replicate the crash at a specific time by editing /app/console/Kernel.php and then restarting the container:

$schedule->command(SystemMaintenance::class)->weeklyOn(0,'12:12')

This causes a crash at 12:12 local time.

If I comment out both calls to Artisan in refreshCache () (SystemMaintenance.php), the server stays up even through a scheduled maintenance. Leaving either one uncommented will still result in a crash. My guess is that the optimize command somehow clears out the app key, even for people who have specified one, leading to a crash.

nwagenmakers commented 9 months ago

@mmomjian nice observation. Is the key stored in the db? Could you check if it is indeed removed after the optimize/optimize:clear is called?

dominikborkowski commented 9 months ago

Just adding 'me too'. I see this issue happen roughly once a week, with scheduled checks set to ever half an hour. 

[2024-01-28 10:23:56] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\\Encryption\\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)
[stacktrace]
mmomjian commented 9 months ago

@nwagenmakers I don't see any reference to the key in the postgres database, but I'm not sure exactly where to look.

te5s3rakt commented 9 months ago

same here (stopps with Error 500) Database is mariadb - restarting web container fixes the issue

Same here. First week with this running, discovered it stopped last night. Restart of the Speedtest container fixed it.

MariaDB here as well. So probably explains why we both can just refresh the speedtest container to get it working, whereas others need to refresh both speedtest and their db containers.

@nwagenmakers I don't see any reference to the key in the postgres database, but I'm not sure exactly where to look.

I crawled mine twice and couldn't find it either.

mmomjian commented 9 months ago

@te5s3rakt I run Postgres and it’s never required a restart, so I’m not sure if that’s related.

RovoMe commented 9 months ago

I have also a 500 on one of my instances, got the following error:

Unsupported cipher or incorrect key length. Supported ciphers are: aes-128-cbc, aes-256-cbc, aes-128-gcm, aes-256-gcm.

I have the same exact error regardless of whether I'm accessing the service via HTTP or HTTPS.

@te5s3rakt I run Postgres and it’s never required a restart, so I’m not sure if that’s related.

I run Postgres as well and face a 500 error when trying to access the service via the browser directly, again regardless whether HTTP or HTTPS is used for browsing the service. Interestingly though, it worked earlier this day for a brief moment where I could see stats on the page, but since then I get the error quoted above.

error

And more surprisingly, the integration into homepage shows an active entry for the service which even updates slightly.

speedtracker speedtest

The problem therefore seems to only affect browser access (tested with Firefox 122.0 and Edge 121.0). The API seems to be working though.

The reported error stems from a TLS handshake failure where the target service called doesn't support the ciphers the client supports. aes-128-cbc indicates a TLS 1.2 secured endpoint. TLS 1.3 i.e. has dropped support of AES_CBC ciphers as they aren't up to modern security standards further. However, AES_GCM ciphers should be usable by TLS 1.3 AFAIK. As this error occurs when accessing the service either via HTTP or HTTPS I guess the cause of this error might be within the service itself. But I haven't looked at the internals of the service yet.

Churator commented 9 months ago

Getting this error too, later version + maria db + API_KEY set as ENV.

@mmomjian nice find. I've tried to run the maintenance commands manually, and seems like it didn't crash the UI. root@02cacc2411dd:/var/www/html# php artisan optimize:clear

INFO Clearing cached bootstrap files.

events .................................................................................................................................. 2ms DONE views ................................................................................................................................... 6ms DONE cache ................................................................................................................................... 2ms DONE route ................................................................................................................................... 1ms DONE config .................................................................................................................................. 1ms DONE compiled ................................................................................................................................ 1ms DONE

root@02cacc2411dd:/var/www/html# php artisan optimize

INFO Caching the framework bootstrap files.

config ................................................................................................................................. 20ms DONE routes ................................................................................................................................. 21ms DONE

I didn't have time to change the schedule (no editor in the docker :) ), but it should be pretty much the same.

p-jones commented 9 months ago

Just to add my 'me too'. Speedtest front end is always visible. Speed tests stop working reliably each Sunday at 0000. The last test is always posted late Saturday night.

alexjustesen commented 8 months ago

For those having an issue, follow the debug steps in this comment on #1035 https://github.com/alexjustesen/speedtest-tracker/issues/1035#issuecomment-1924907035

mmomjian commented 8 months ago

@alexjustesen here are my results. The cache driver is set to file. I installed using the docker compose instructions at the website. Are we supposed to use the .env.example file from the github repo for the /config folder? My .env file is autogenerated as the instructions don't actually specify to create a .env file. 

# php artisan about

  Environment ................................................................
  Application Name ......................................... Speedtest Tracker
  Laravel Version .................................................... 10.40.0
  PHP Version ......................................................... 8.2.14
  Composer Version ..................................................... 2.6.6
  Environment ..................................................... production
  Debug Mode ............................................................. OFF
  URL .............................................................. localhost
  Maintenance Mode ....................................................... OFF

  Cache ......................................................................
  Config .............................................................. CACHED
  Events .......................................................... NOT CACHED
  Routes .............................................................. CACHED
  Views ............................................................... CACHED

  Drivers ....................................................................
  Broadcasting ........................................................... log
  Cache ................................................................. file
  Database ............................................................. pgsql
  Logs ................................................................ stderr
  Mail .................................................................. smtp
  Queue ............................................................. database
  Session ........................................................... database

  Filament ...................................................................
  Packages ................... filament, forms, notifications, support, tables
  Version ............................................................ v3.1.47
  Views ........................................................ NOT PUBLISHED

  Livewire ...................................................................
  Livewire ............................................................ v3.3.5

  Speedtest Tracker ..........................................................
  Out of date ............................................................ Yes
  Version ............................................................ v0.14.4

/config/.env:

# A full list of options can be found in the '.env.example' file.

# Application key
# Used for encryption where needed, if a key is not generated run: php artisan key:generate
APP_KEY=base64:<redacted>

# Application URL
# Not used in the URL but effects links in emails and notifications: https://docs.speedtest-tracker.dev/faqs#links-in-emails-dont-point-to-the-correct-url
APP_URL=http://localhost

# SMTP mail config
# Only SMTP protocol is supported at this time: https://docs.speedtest-tracker.dev/settings/notifications/mail
MAIL_MAILER=smtp
MAIL_HOST=mailhog
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="Speedtest Tracker"

# Telegram bot config
TELEGRAM_BOT_TOKEN=null
alexjustesen commented 8 months ago

@mmomjian correct, when you start up the container if a .env file doesn't exist it's copied from .env.production.

Start up code:

https://github.com/alexjustesen/speedtest-tracker/blob/main/docker/deploy/etc/s6-overlay/scripts/laravel-automations#L46-L53

If you change anything in this file make sure to restart your container so the changes are picked up. I can see your cache driver is set to file, because we don't mount the storage directory with the correct permissions you'll want to use the database cache driver.

mmomjian commented 8 months ago

@alexjustesen I am still on v0.14.4, I believe the bug you had linked to may be a different issue, sorry for the confusion on my end. The issue in this thread is a crash every Sunday morning at 00:00 local time. Do you want me to try setting the cache driver to database and see if it stays up this week? What is the correct environment variable to set to change the cache driver?

alexjustesen commented 8 months ago

@alexjustesen I am still on v0.14.4, I believe the bug you had linked to may be a different issue, sorry for the confusion on my end. The issue in this thread is a crash every Sunday morning at 00:00 local time. Do you want me to try setting the cache driver to database and see if it stays up this week? What is the correct environment variable to set to change the cache driver?

Please do, it should be set to database.

mmomjian commented 8 months ago

I added to CACHE_DRIVER: 'database' to the docker compose file and I now get:

  Drivers ..........................................................................................................................................
  Broadcasting ................................................................................................................................. log
  Cache ................................................................................................................................... database

I'll report back on Sunday.

Churator commented 8 months ago

For those having an issue, follow the debug steps in this comment on #1035 #1035 (comment)

Environment ...................................................................................................................................... Application Name ............................................................................................................... Speedtest Tracker Laravel Version .......................................................................................................................... 10.40.0 PHP Version ............................................................................................................................... 8.2.14 Composer Version ........................................................................................................................... 2.6.6 Environment ........................................................................................................................... production Debug Mode ................................................................................................................................... OFF URL ............................................................................................................................. spt.i.home.org Maintenance Mode ............................................................................................................................. OFF

Cache ............................................................................................................................................ Config .................................................................................................................................... CACHED Events ................................................................................................................................ NOT CACHED Routes .................................................................................................................................... CACHED Views ..................................................................................................................................... CACHED

Drivers .......................................................................................................................................... Broadcasting ................................................................................................................................. log Cache ....................................................................................................................................... file Database ................................................................................................................................... mysql Logs ...................................................................................................................................... stderr Mail ........................................................................................................................................ smtp Queue ................................................................................................................................... database Session ................................................................................................................................. database

Filament ......................................................................................................................................... Packages ......................................................................................... filament, forms, notifications, support, tables Version .................................................................................................................................. v3.1.47 Views .............................................................................................................................. NOT PUBLISHED

Livewire ......................................................................................................................................... Livewire .................................................................................................................................. v3.3.5

Speedtest Tracker ................................................................................................................................ Out of date .................................................................................................................................. Yes Version .................................................................................................................................. v0.14.4

Docker-compose speedtest: container_name: speedtest image: ghcr.io/alexjustesen/speedtest-tracker:latest volumes:

Documentation say the default file cache is probably best in most cases, why database then ?

cgladuefox commented 8 months ago

stopping the container and restarting fixes the issue

Starfiresg1 commented 8 months ago

I can confirm this is happening every Sunday - probably exactly at 0:00 - restarting the container fixes this as reported. It just happened again - I've now set up a cron job at 0:01 every Sunday that restarts the container as a workaround.

  Environment ......................................................................................................................................  
  Application Name ............................................................................................................... Speedtest Tracker  
  Laravel Version .......................................................................................................................... 10.43.0  
  PHP Version ............................................................................................................................... 8.2.15  
  Composer Version ........................................................................................................................... 2.6.6  
  Environment ........................................................................................................................... production  
  Debug Mode ................................................................................................................................... OFF  
  URL .................................................................................................................................... localhost  
  Maintenance Mode ............................................................................................................................. OFF  

  Cache ............................................................................................................................................  
  Config .................................................................................................................................... CACHED  
  Events ................................................................................................................................ NOT CACHED  
  Routes .................................................................................................................................... CACHED  
  Views ..................................................................................................................................... CACHED  

  Drivers ..........................................................................................................................................  
  Broadcasting ................................................................................................................................. log  
  Cache ................................................................................................................................... database  
  Database ................................................................................................................................... mysql  
  Logs ...................................................................................................................................... stderr  
  Mail ........................................................................................................................................ smtp  
  Queue ................................................................................................................................... database  
  Session ................................................................................................................................. database  

  Filament .........................................................................................................................................  
  Packages ......................................................................................... filament, forms, notifications, support, tables  
  Version .................................................................................................................................. v3.2.23  
  Views .............................................................................................................................. NOT PUBLISHED  

  Livewire .........................................................................................................................................  
  Livewire .................................................................................................................................. v3.4.4  

  Speedtest Tracker ................................................................................................................................  
  Out of date ................................................................................................................................... No  
  Version .................................................................................................................................. v0.14.6

The exact moment the error occurs the webpage will only display error 500 and the following appears in the log continuously until the container is restartet: (offset from 0:00 by 1 hour due to timezones)

[2024-02-03 23:00:12] production.ERROR: No application encryption key has been specified. {"exception":"[object] (Illuminate\\Encryption\\MissingAppKeyException(code: 0): No application encryption key has been specified. at /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php:79)
      
[stacktrace]
#0 /var/www/html/vendor/laravel/framework/src/Illuminate/Support/helpers.php(307): Illuminate\\Encryption\\EncryptionServiceProvider->Illuminate\\Encryption\\{closure}()
#1 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(77): tap()
#2 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(60): Illuminate\\Encryption\\EncryptionServiceProvider->key()
#3 /var/www/html/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php(32): Illuminate\\Encryption\\EncryptionServiceProvider->parseKey()
#4 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(908): Illuminate\\Encryption\\EncryptionServiceProvider->Illuminate\\Encryption\\{closure}()
#5 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\\Container\\Container->build()
#6 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\\Container\\Container->resolve()
#7 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\\Foundation\\Application->resolve()
#8 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\\Container\\Container->make()
#9 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(1066): Illuminate\\Foundation\\Application->make()
#10 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(982): Illuminate\\Container\\Container->resolveClass()
#11 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(943): Illuminate\\Container\\Container->resolveDependencies()
#12 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(795): Illuminate\\Container\\Container->build()
#13 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(957): Illuminate\\Container\\Container->resolve()
#14 /var/www/html/vendor/laravel/framework/src/Illuminate/Container/Container.php(731): Illuminate\\Foundation\\Application->resolve()
#15 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Application.php(942): Illuminate\\Container\\Container->make()
#16 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(172): Illuminate\\Foundation\\Application->make()
#17 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#18 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(805): Illuminate\\Pipeline\\Pipeline->then()
#19 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(784): Illuminate\\Routing\\Router->runRouteWithinStack()
#20 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(748): Illuminate\\Routing\\Router->runRoute()
#21 /var/www/html/vendor/laravel/framework/src/Illuminate/Routing/Router.php(737): Illuminate\\Routing\\Router->dispatchToRoute()
#22 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(200): Illuminate\\Routing\\Router->dispatch()
#23 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}()
#24 /var/www/html/vendor/livewire/livewire/src/Features/SupportDisablingBackButtonCache/DisableBackButtonCacheMiddleware.php(19): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#25 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Livewire\\Features\\SupportDisablingBackButtonCache\\DisableBackButtonCacheMiddleware->handle()
#26 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#27 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ConvertEmptyStringsToNull->handle()
#28 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php(36): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#29 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\TrimStrings->handle()
#30 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#31 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle()
#32 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(99): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#33 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle()
#34 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#35 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\HandleCors->handle()
#36 /var/www/html/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php(39): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#37 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\\Http\\Middleware\\TrustProxies->handle()
#38 /var/www/html/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}()
#39 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(175): Illuminate\\Pipeline\\Pipeline->then()
#40 /var/www/html/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(144): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter()
#41 /var/www/html/public/index.php(51): Illuminate\\Foundation\\Http\\Kernel->handle()
#42 {main}
"}

It seams every Sunday at 0:00 the app looeses it's app key - even through it is set by the environment variable in docker and recognized in the log on startup:

🔒 SSL_MODE has been set to MIXED, setting the web server to work in HTTP + HTTPS...
🐇  Configuring Speedtest Tracker...
🔒  Fixing app path file permissions...
✅  Permissions fixed.
✅  Environment file exists.
🔗  Creating symlinks for config and log files...
✅  Symlinks created.
✅  An application key environment variable exists.
💰  Building the cache...
✅  Cache set.
🚛  Migrating the database...
✅  Database migrated.
✅  All set, Speedtest Tracker started.
[03-Feb-2024 22:35:59] NOTICE: fpm is running, pid 117
[03-Feb-2024 22:35:59] NOTICE: ready to handle connections
[03-Feb-2024 22:35:59] NOTICE: systemd monitor interval set to 10000ms

As the log shows I restartet the container about 30 minutes before midnight (due to updating the image to the latest version). Exactly at midnight it errored out - less than an hour after starting it. So the amount the time the container was running doesn't seem to matter at all - it will stop working at Sunday 0:00 regardless of how long it was running before (mere minutes or a week).

Rahzadan commented 8 months ago

For me, midnight on Sunday was just over an hour ago as of this post, and sure enough, I couldn't access the container. It was working fine earlier today (Saturday). I hasn't noticed this before, but I was checking the container's stability after I'd figured out the issue referenced in #1087. I simply restarted the container and all is well again.

There's definitely something still wrong here.

Exit24 commented 8 months ago

I’m EST and it was midnight 39 minutes ago and that’s when it stopped working for me again too. A reboot fixes it.

mmomjian commented 8 months ago

Died at 12:00am local time. It seems that the cache was reset to file. A restart of the Docker container fixes the issue, and it now reports Cache...database again.

This output is at the time of the crash, before container restart.

# php artisan about

  Environment .....................................................................................................
  Application Name .............................................................................. Speedtest Tracker
  Laravel Version ......................................................................................... 10.40.0
  PHP Version .............................................................................................. 8.2.14
  Composer Version .......................................................................................... 2.6.6
  Environment .......................................................................................... production
  Debug Mode .................................................................................................. OFF
  URL ................................................................................................... localhost
  Maintenance Mode ............................................................................................ OFF

  Cache ...........................................................................................................
  Config ................................................................................................... CACHED
  Events ............................................................................................... NOT CACHED
  Routes ................................................................................................... CACHED
  Views .................................................................................................... CACHED

  Drivers .........................................................................................................
  Broadcasting ................................................................................................ log
  Cache ...................................................................................................... file
  Database .................................................................................................. mysql
  Logs ..................................................................................................... stderr
  Mail ....................................................................................................... smtp
  Queue .................................................................................................. database
  Session ................................................................................................ database

  Filament ........................................................................................................
  Packages ........................................................ filament, forms, notifications, support, tables
  Version ................................................................................................. v3.1.47
  Views ............................................................................................. NOT PUBLISHED

  Livewire ........................................................................................................
  Livewire ................................................................................................. v3.3.5

  Speedtest Tracker ...............................................................................................
  Out of date ................................................................................................. Yes
  Version ................................................................................................. v0.14.4