search

truecharts / public

Community Helm Chart Repository
https://truecharts.org
GNU Affero General Public License v3.0
1.13k stars 617 forks source link

Nextcloud regularly exceeds PostgreSQL max_connections limit #10032

Closed justsomescripts closed 9 months ago

justsomescripts commented 1 year ago

App Name

nextcloud

SCALE Version

22.12.3.1

App Version

27.0.0_21.0.7

Application Events

No relevant events (no new pods scheduled etc.)

Application Logs

[index] Error: Doctrine\DBAL\Exception: Failed to connect to the database: An exception occurred in the driver: SQLSTATE[08006] [7] FATAL:  remaining connection slots are reserved for non-replication superuser connections at <<closure>>

 0. /var/www/html/3rdparty/doctrine/dbal/src/Connection.php line 1531
    OC\DB\Connection->connect()
 1. /var/www/html/3rdparty/doctrine/dbal/src/Connection.php line 1029
    Doctrine\DBAL\Connection->getWrappedConnection()
 2. /var/www/html/lib/private/DB/Connection.php line 262
    Doctrine\DBAL\Connection->executeQuery("SELECT * FROM \"oc_appconfig\"", [], [], null)
 3. /var/www/html/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php line 345
    OC\DB\Connection->executeQuery("SELECT * FROM \"oc_appconfig\"", [], [])
 4. /var/www/html/lib/private/DB/QueryBuilder/QueryBuilder.php line 280
    Doctrine\DBAL\Query\QueryBuilder->execute()
 5. /var/www/html/lib/private/AppConfig.php line 418
    OC\DB\QueryBuilder\QueryBuilder->execute()
 6. /var/www/html/lib/private/AppConfig.php line 184
    OC\AppConfig->loadConfigValues()
 7. /var/www/html/lib/private/AppConfig.php line 374
    OC\AppConfig->getApps()
 8. /var/www/html/lib/private/legacy/OC_App.php line 803
    OC\AppConfig->getValues(false, "installed_version")
 9. /var/www/html/lib/private/Server.php line 733
    OC_App::getAppVersions()
10. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 171
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
11. /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php line 122
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
12. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 138
    Pimple\Container->offsetGet("OC\\Memcache\\Factory")
13. /var/www/html/lib/private/ServerContainer.php line 171
    OC\AppFramework\Utility\SimpleContainer->query("OC\\Memcache\\Factory", true)
14. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 65
    OC\ServerContainer->query("OC\\Memcache\\Factory")
15. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 193
    OC\AppFramework\Utility\SimpleContainer->get("OC\\Memcache\\Factory")
16. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 171
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
17. /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php line 118
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
18. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 138
    Pimple\Container->offsetGet("OCP\\ICacheFactory")
19. /var/www/html/lib/private/ServerContainer.php line 171
    OC\AppFramework\Utility\SimpleContainer->query("OCP\\ICacheFactory", true)
20. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 65
    OC\ServerContainer->query("OCP\\ICacheFactory")
21. /var/www/html/lib/private/Server.php line 1115
    OC\AppFramework\Utility\SimpleContainer->get("OCP\\ICacheFactory")
22. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 171
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
23. /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php line 122
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
24. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 138
    Pimple\Container->offsetGet("OCP\\Lock\\ILockingProvider")
25. /var/www/html/lib/private/ServerContainer.php line 171
    OC\AppFramework\Utility\SimpleContainer->query("OCP\\Lock\\ILockingProvider", true)
26. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 65
    OC\ServerContainer->query("OCP\\Lock\\ILockingProvider")
27. /var/www/html/lib/private/Server.php line 2067
    OC\AppFramework\Utility\SimpleContainer->get("OCP\\Lock\\ILockingProvider")
28. /var/www/html/lib/private/Files/View.php line 106
    OC\Server->getLockingProvider()
29. /var/www/html/lib/private/Server.php line 467
    OC\Files\View->__construct()
30. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 171
    OC\Server->OC\{closure}("*** sensitive parameters replaced ***")
31. /var/www/html/3rdparty/pimple/pimple/src/Pimple/Container.php line 122
    OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility\{closure}("*** sensitive parameters replaced ***")
32. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 138
    Pimple\Container->offsetGet("OC\\Files\\Node\\HookConnector")
33. /var/www/html/lib/private/ServerContainer.php line 171
    OC\AppFramework\Utility\SimpleContainer->query("OC\\Files\\Node\\HookConnector", true)
34. /var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php line 65
    OC\ServerContainer->query("OC\\Files\\Node\\HookConnector")
35. /var/www/html/lib/private/Server.php line 1469
    OC\AppFramework\Utility\SimpleContainer->get("OC\\Files\\Node\\HookConnector")
36. /var/www/html/lib/base.php line 631
    OC\Server->boot()
37. /var/www/html/lib/base.php line 1173
    OC::init()
38. /var/www/html/index.php line 34
    require_once("/var/www/html/lib/base.php")

GET /apps/photos/api/v1/preview/1933527?etag=1a20718eaca4f91ed07f5f2a79ffef73&x=64&y=64
from 192.168.178.10 at 2023-06-29T22:55:04+02:00

Application Configuration

Type: Deployment
Replicas: 1
Default Phone Region: DE
Access IP: <Server LAN IP>
Max Chunk Size: 52428800
Enable Previews: yes
Enable Imaginary: no
Enable cron: yes
Cron Schedule: */30 * * * *
Max X: 4096
Max Y: 4096
Max Memory: 4096
Max File Size Image: 512
JPEG Quality: 90
Square Sizes: 64 256 1024
Width Sizes: 64 256 1024 4096
Height Sizes: 64 256 1024 4096
Providers: GIF, JPEG, PNG, HEIC. Movie
Notify Push: yes
Memory Limit: 4G
Upload Limit: 128G
Max Children: 128
Start Servers: 16
Minimum Spare Servers: 8
Maximum Spare Servers: 16
Service Type: ClusterIP + Ingress

Rest is unchanged. Also tried with default php-fpm settings!

Describe the bug

When scrolling through Nextcloud Photos, Nextcloud hangs and reaches the max_connections limit of PostgreSQL. This happens with default settings for php-fpm, as well as with my custom ones. When querying the Nextcloud database fusing SHOW max_connections;, the return value is 100. I did this from the bootstrap-controller container on nextcloud-cnpg-main-1:

$ /usr/bin/psql
psql (15.2 (Debian 15.2-1.pgdg110+1))
Type "help" for help.

postgres=# SHOW max_connections;
 max_connections 
-----------------
 100
(1 row)

postgres=#

To Reproduce

  1. Open Nextcloud Webinterface
  2. Go to the Nextcloud Photos tab
  3. Scroll trough photos

Expected Behavior

Nextcloud doesn't crash because of the max_connection limit

Screenshots

image

Additional Context

This doesn't happen when heavily decreasing the max_children php-fpm limit, but then it crashes when reaching max_children which makes it just as bad.

I've read and agree with the following

PrivatePuffin commented 1 year ago

Interesting, the pooler should prevent this imho.

Tangado-Haid commented 12 months ago

I got the same issue Nextcloud 27.1.1_22.0.39 Scale TrueNAS-SCALE-22.12.4.2

stavros-k commented 12 months ago

I got the same issue

Thank you for making making that comment, that helped me to find the problem and solve it.

stavros-k commented 9 months ago

AFter the rework, pooler should take care of this. If not please re open