Navigating in the categories tree leads to errors in Redis: "Redis session exceeded concurrent connections"

Preconditions and environment

v.2.4.7-p1
Redis enabled

Steps to reproduce

Go to the backend
Navigate to Manage Categories
Add categories until you have a huge category tree with many subordinate categories
Open some child-nodes in categories tree by clicking on the arrow next to the category in the tree
Watch the logs and network logs

Expected result

No error occurs when navigating through the categories tree in the admin.

Actual result

About half of the requests fail:

curl 'https://www.shop.com/minad/catalog/category/categoriesJson/key/cbba38596330878bdc5085ede5a0244bcb55e944d2c46ced02f103084a3fc2cd/?isAjax=true' \
  --data-raw 'id=25466&store=0&form_key=3cM2Gt4PVMHQFtTM'

=> 500 Internal Server Error

With the following message in exception log:

Stack trace:
#0 /var/www/html/web/vendor/magento/framework/Session/SaveHandler/Redis.php(84): Cm\RedisSession\Handler->read()
#1 /var/www/html/web/vendor/magento/framework/Session/SaveHandler.php(213): Magento\Framework\Session\SaveHandler\Redis->read()
#2 /var/www/html/web/vendor/magento/framework/Session/SaveHandler.php(126): Magento\Framework\Session\SaveHandler->callSafely()
#3 [internal function]: Magento\Framework\Session\SaveHandler->read()
#4 /var/www/html/web/vendor/magento/framework/Session/SessionManager.php(187): session_start()
#5 /var/www/html/web/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(149): Magento\Framework\Session\SessionManager->start()
#6 /var/www/html/web/vendor/magento/framework/Session/SessionManager.php(124): Magento\Backend\Model\Auth\Session\Interceptor->start()
#7 /var/www/html/web/vendor/magento/module-backend/Model/Auth/Session.php(104): Magento\Framework\Session\SessionManager->__construct()
#8 /var/www/html/web/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(14): Magento\Backend\Model\Auth\Session->__construct()
#9 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Backend\Model\Auth\Session\Interceptor->__construct()
#10 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#11 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#12 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#13 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#14 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#15 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#16 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(125): Magento\Framework\ObjectManager\Factory\Compiled->get()
#17 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(86): Magento\Framework\ObjectManager\Factory\Compiled->parseArray()
#18 /var/www/html/web/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#19 /var/www/html/web/vendor/magento/framework/App/FrontController.php(109): Magento\Framework\ObjectManager\ObjectManager->get()
#20 /var/www/html/web/generated/code/Magento/Framework/App/FrontController/Interceptor.php(14): Magento\Framework\App\FrontController->__construct()
#21 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Framework\App\FrontController\Interceptor->__construct()
#22 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#23 /var/www/html/web/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#24 /var/www/html/web/vendor/magento/framework/App/Http.php(115): Magento\Framework\ObjectManager\ObjectManager->get()
#25 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Framework\App\Http->launch()
#26 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\App\Http\Interceptor->___callParent()
#27 /var/www/html/web/vendor/magento/module-application-performance-monitor/Plugin/ApplicationPerformanceMonitor.php(38): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#28 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(135): Magento\ApplicationPerformanceMonitor\Plugin\ApplicationPerformanceMonitor->aroundLaunch()
#29 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#30 /var/www/html/web/generated/code/Magento/Framework/App/Http/Interceptor.php(23): Magento\Framework\App\Http\Interceptor->___callPlugins()
#31 /var/www/html/web/vendor/magento/framework/App/Bootstrap.php(264): Magento\Framework\App\Http\Interceptor->launch()
#32 /var/www/html/web/pub/index.php(30): Magento\Framework\App\Bootstrap->run()
#33 {main}

Next Magento\Framework\Exception\LocalizedException: Redis session exceeded concurrent connections in /var/www/html/web/vendor/magento/framework/Session/SaveHandler/Redis.php:86
Stack trace:
#0 /var/www/html/web/vendor/magento/framework/Session/SaveHandler.php(213): Magento\Framework\Session\SaveHandler\Redis->read()
#1 /var/www/html/web/vendor/magento/framework/Session/SaveHandler.php(126): Magento\Framework\Session\SaveHandler->callSafely()
#2 [internal function]: Magento\Framework\Session\SaveHandler->read()
#3 /var/www/html/web/vendor/magento/framework/Session/SessionManager.php(187): session_start()
#4 /var/www/html/web/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(149): Magento\Framework\Session\SessionManager->start()
#5 /var/www/html/web/vendor/magento/framework/Session/SessionManager.php(124): Magento\Backend\Model\Auth\Session\Interceptor->start()
#6 /var/www/html/web/vendor/magento/module-backend/Model/Auth/Session.php(104): Magento\Framework\Session\SessionManager->__construct()
#7 /var/www/html/web/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(14): Magento\Backend\Model\Auth\Session->__construct()
#8 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Backend\Model\Auth\Session\Interceptor->__construct()
#9 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#10 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#11 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#12 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#13 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#14 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#15 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(125): Magento\Framework\ObjectManager\Factory\Compiled->get()
#16 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(86): Magento\Framework\ObjectManager\Factory\Compiled->parseArray()
#17 /var/www/html/web/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#18 /var/www/html/web/vendor/magento/framework/App/FrontController.php(109): Magento\Framework\ObjectManager\ObjectManager->get()
#19 /var/www/html/web/generated/code/Magento/Framework/App/FrontController/Interceptor.php(14): Magento\Framework\App\FrontController->__construct()
#20 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Framework\App\FrontController\Interceptor->__construct()
#21 /var/www/html/web/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#22 /var/www/html/web/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#23 /var/www/html/web/vendor/magento/framework/App/Http.php(115): Magento\Framework\ObjectManager\ObjectManager->get()
#24 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Framework\App\Http->launch()
#25 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\App\Http\Interceptor->___callParent()
#26 /var/www/html/web/vendor/magento/module-application-performance-monitor/Plugin/ApplicationPerformanceMonitor.php(38): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#27 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(135): Magento\ApplicationPerformanceMonitor\Plugin\ApplicationPerformanceMonitor->aroundLaunch()
#28 /var/www/html/web/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#29 /var/www/html/web/generated/code/Magento/Framework/App/Http/Interceptor.php(23): Magento\Framework\App\Http\Interceptor->___callPlugins()
#30 /var/www/html/web/vendor/magento/framework/App/Bootstrap.php(264): Magento\Framework\App\Http\Interceptor->launch()
#31 /var/www/html/web/pub/index.php(30): Magento\Framework\App\Bootstrap->run()
#32 {main} {"report_id":"9ab6e212dd05f9cecc56387e18e3ab7bdedc521ec4958b4a3e6564a8f5c49011","exception":"[object] (Magento\\Framework\\Exception\\LocalizedException(code: 0): Redis session exceeded concurrent connections at /var/www/html/web/vendor/magento/framework/Session/SaveHandler/Redis.php:86)
[previous exception] [object] (Cm\\RedisSession\\ConcurrentConnectionsExceededException(code: 0):  at /var/www/html/web/vendor/colinmollenhour/php-redis-session-abstract/src/Cm/RedisSession/Handler.php:531)"} []

Additional information

No response

Release note

No response

Triage and priority

[ ] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
[ ] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
[X] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
[ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
[ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

Hi @netzkollektiv. Thank you for your report. To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release
For more details, review the Magento Contributor Assistant documentation.
Add a comment to assign the issue: @magento I am working on this
To learn more about issue processing workflow, refer to the Code Contributions.

Join Magento Community Engineering Slack and ask your questions in #github channel. :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting. :clock10: You can find the schedule on the Magento Community Calendar page. :telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

Hi @engcom-Delta. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue.
[ ] 3. Add Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
[ ] 4. Verify that the issue is reproducible on 2.4-develop branch
Details
- Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
[ ] 5. Add label Issue: Confirmed once verification is complete.
[ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

Also running into this problem over here. In our case it not only happens when clicking the arrow, but also when just clicking a deeper level of category itself, in that case it's even worse, we sometimes see > 70 similar ajax requests happening simultaneous of which a lot are failing. One of our shops with this problem is hosted on hypernode.com and we notice we get a bunch of HTTP status codes 429 back, because their rate limiter sees way too many requests happening at the same time from the same source IP.

I think the problem got introduced by https://github.com/magento/magento2/commit/67f1b8f0c89cc83302bc7781a10dca931b8e7097 which was released in Magento 2.4.7 and higher, more specifically in this piece of code.

I'm not exactly sure what this code is supposed to do, but it sounds wrong to me to perform tens if not hundreds of similar ajax requests for info when you can try to get the necessary info as well within a single request.

Maybe @Atul-glo35265 is around to try to explain how that code is supposed to work and if there isn't a way to optimise it so it doesn't send all those ajax requests in a loop, but bundle them up in a single request somehow? /cc @kandy

@hostep Are you sure that this issue is not reproducible on version < 2.4.7? My assumption that this issue caused by switching from http1 to http2 that allow up to 100 parallel request to server

Usually BFF (like graphql) responsibly for batching requests, so implementing batching like /tree/child/?ids=[1,2,3,4] is optimal for now

This happens in 2.4.7 and 2.4.7-p1 for me Lot of XHR requests coming to PageBuilder previews preview2 preview1

Stack trace:

[2024-06-25T15:32:16.627508+00:00] main.CRITICAL: Cm\RedisSession\ConcurrentConnectionsExceededException in /home/www/vhosts/magento/vendor/colinmollenhour/php-redis-session-abstract/src/Cm/RedisSession/Handler.php:531
Stack trace:
#0 /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler/Redis.php(84): Cm\RedisSession\Handler->read()
#1 /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler.php(213): Magento\Framework\Session\SaveHandler\Redis->read()
#2 /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler.php(126): Magento\Framework\Session\SaveHandler->callSafely()
#3 [internal function]: Magento\Framework\Session\SaveHandler->read()
#4 /home/www/vhosts/magento/vendor/magento/framework/Session/SessionManager.php(187): session_start()
#5 /home/www/vhosts/magento/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(32): Magento\Framework\Session\SessionManager->start()
#6 /home/www/vhosts/magento/vendor/magento/framework/Session/SessionManager.php(124): Magento\Backend\Model\Auth\Session\Interceptor->start()
#7 /home/www/vhosts/magento/vendor/magento/module-backend/Model/Auth/Session.php(104): Magento\Framework\Session\SessionManager->__construct()
#8 /home/www/vhosts/magento/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(14): Magento\Backend\Model\Auth\Session->__construct()
#9 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Backend\Model\Auth\Session\Interceptor->__construct()
#10 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#11 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#12 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#13 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#14 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#15 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#16 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(125): Magento\Framework\ObjectManager\Factory\Compiled->get()
#17 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(86): Magento\Framework\ObjectManager\Factory\Compiled->parseArray()
#18 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#19 /home/www/vhosts/magento/vendor/magento/framework/App/FrontController.php(109): Magento\Framework\ObjectManager\ObjectManager->get()
#20 /home/www/vhosts/magento/generated/code/Magento/Framework/App/FrontController/Interceptor.php(14): Magento\Framework\App\FrontController->__construct()
#21 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Framework\App\FrontController\Interceptor->__construct()
#22 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#23 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#24 /home/www/vhosts/magento/vendor/magento/framework/App/Http.php(115): Magento\Framework\ObjectManager\ObjectManager->get()
#25 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Framework\App\Http->launch()
#26 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\App\Http\Interceptor->___callParent()
#27 /home/www/vhosts/magento/vendor/magento/module-application-performance-monitor/Plugin/ApplicationPerformanceMonitor.php(38): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#28 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(135): Magento\ApplicationPerformanceMonitor\Plugin\ApplicationPerformanceMonitor->aroundLaunch()
#29 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#30 /home/www/vhosts/magento/generated/code/Magento/Framework/App/Http/Interceptor.php(23): Magento\Framework\App\Http\Interceptor->___callPlugins()
#31 /home/www/vhosts/magento/vendor/magento/framework/App/Bootstrap.php(264): Magento\Framework\App\Http\Interceptor->launch()
#32 /home/www/vhosts/magento/pub/index.php(30): Magento\Framework\App\Bootstrap->run()
#33 {main}

Next Magento\Framework\Exception\LocalizedException: Redis session exceeded concurrent connections in /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler/Redis.php:86
Stack trace:
#0 /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler.php(213): Magento\Framework\Session\SaveHandler\Redis->read()
#1 /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler.php(126): Magento\Framework\Session\SaveHandler->callSafely()
#2 [internal function]: Magento\Framework\Session\SaveHandler->read()
#3 /home/www/vhosts/magento/vendor/magento/framework/Session/SessionManager.php(187): session_start()
#4 /home/www/vhosts/magento/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(32): Magento\Framework\Session\SessionManager->start()
#5 /home/www/vhosts/magento/vendor/magento/framework/Session/SessionManager.php(124): Magento\Backend\Model\Auth\Session\Interceptor->start()
#6 /home/www/vhosts/magento/vendor/magento/module-backend/Model/Auth/Session.php(104): Magento\Framework\Session\SessionManager->__construct()
#7 /home/www/vhosts/magento/generated/code/Magento/Backend/Model/Auth/Session/Interceptor.php(14): Magento\Backend\Model\Auth\Session->__construct()
#8 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Backend\Model\Auth\Session\Interceptor->__construct()
#9 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#10 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#11 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#12 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#13 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(79): Magento\Framework\ObjectManager\Factory\Compiled->get()
#14 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(150): Magento\Framework\ObjectManager\Factory\Compiled->create()
#15 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(125): Magento\Framework\ObjectManager\Factory\Compiled->get()
#16 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(86): Magento\Framework\ObjectManager\Factory\Compiled->parseArray()
#17 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#18 /home/www/vhosts/magento/vendor/magento/framework/App/FrontController.php(109): Magento\Framework\ObjectManager\ObjectManager->get()
#19 /home/www/vhosts/magento/generated/code/Magento/Framework/App/FrontController/Interceptor.php(14): Magento\Framework\App\FrontController->__construct()
#20 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/AbstractFactory.php(121): Magento\Framework\App\FrontController\Interceptor->__construct()
#21 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/Factory/Compiled.php(108): Magento\Framework\ObjectManager\Factory\AbstractFactory->createObject()
#22 /home/www/vhosts/magento/vendor/magento/framework/ObjectManager/ObjectManager.php(73): Magento\Framework\ObjectManager\Factory\Compiled->create()
#23 /home/www/vhosts/magento/vendor/magento/framework/App/Http.php(115): Magento\Framework\ObjectManager\ObjectManager->get()
#24 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(58): Magento\Framework\App\Http->launch()
#25 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(138): Magento\Framework\App\Http\Interceptor->___callParent()
#26 /home/www/vhosts/magento/vendor/magento/module-application-performance-monitor/Plugin/ApplicationPerformanceMonitor.php(38): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#27 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(135): Magento\ApplicationPerformanceMonitor\Plugin\ApplicationPerformanceMonitor->aroundLaunch()
#28 /home/www/vhosts/magento/vendor/magento/framework/Interception/Interceptor.php(153): Magento\Framework\App\Http\Interceptor->Magento\Framework\Interception\{closure}()
#29 /home/www/vhosts/magento/generated/code/Magento/Framework/App/Http/Interceptor.php(23): Magento\Framework\App\Http\Interceptor->___callPlugins()
#30 /home/www/vhosts/magento/vendor/magento/framework/App/Bootstrap.php(264): Magento\Framework\App\Http\Interceptor->launch()
#31 /home/www/vhosts/magento/pub/index.php(30): Magento\Framework\App\Bootstrap->run()
#32 {main} {"report_id":"054a95716720fcc93be4f07fb1d91d9ae7089fd62f426b0feb7ebf9aca5fd7ee","exception":"[object] (Magento\\Framework\\Exception\\LocalizedException(code: 0): Redis session exceeded concurrent connections at /home/www/vhosts/magento/vendor/magento/framework/Session/SaveHandler/Redis.php:86)
[previous exception] [object] (Cm\\RedisSession\\ConcurrentConnectionsExceededException(code: 0):  at /home/www/vhosts/magento/vendor/colinmollenhour/php-redis-session-abstract/src/Cm/RedisSession/Handler.php:531)"} []

Catalog has about 3k of categories and single store in my case Redis (almost default) config from app/etc/env.php:

'session' => [
        'save' => 'redis',
        'redis' => [
            'host' => 'localhost',
            'port' => '6379',
            'password' => '',
            'timeout' => '2.5',
            'persistent_identifier' => '',
            'database' => '3',
            'compression_threshold' => '2048',
            'compression_library' => 'gzip',
            'log_level' => '4',
            'max_concurrency' => '6',
            'break_after_frontend' => '5',
            'break_after_adminhtml' => '30',
            'first_lifetime' => '600',
            'bot_first_lifetime' => '60',
            'bot_lifetime' => '7200',
            'disable_locking' => '0',
            'min_lifetime' => '60',
            'max_lifetime' => '2592000',
            'sentinel_master' => '',
            'sentinel_servers' => '',
            'sentinel_connect_retries' => '5',
            'sentinel_verify_master' => '0'
        ]
    ],

Doubt that setting bigger value for max_concurrency is a good idea, but can be a workaround

@hostep Are you sure that this issue is not reproducible on version < 2.4.7? My assumption that this issue caused by switching from http1 to http2 that allow up to 100 parallel request to server Usually BFF (like graphql) responsibly for batching requests, so implementing batching like /tree/child/?ids=[1,2,3,4] is optimal for now

I can double check with 2.4.6, but since the category tree code was refactored in 2.4.7, I don't think it will trigger on anything lower than 2.4.7. Simultaneous request with HTTP/2 is good for dealing with static assets by the browser (images, scripts, css, ...). So one connection that fetches the html (outputted by a single php process) and then fetching all the other assets simultaneously over 1 and the same TCP connection (which have nothing to do with php usually). But in this case, you are sending tens if not hundreds of ajax requests (almost) simultaneously, which the server than has to start processing with php, and validating the sessions for each request to make sure you are authenticated and then fetching, per tiny request, the little info it needs. This seems not a good behavior in my opinion, way too much overhead, way too many php processes that are doing things. Just fetch the info you need with one single ajax request and deal with the big response in the frontend js code.

As far as batching goes, I don't think that happens here at the moment. If you indeed would batch like 50 things in a single request, that should significantly improve the situation.

@lamskoy: could you open a new issue for your problem? It may sound like the same issue at a glance, but it definitely isn't the same problem (unless those preview calls are generated by the ajax calls discussed in this issue?).

Hi @netzkollektiv ,

Verified the issue on 2.4-develop and 2.4.7 and we are not able to reproduce this issue.

Preconditions and environment

1.Redis enabled

Steps to reproduce:-

1.Go to the backend 2.Navigate to Manage Categories 3.Open some child-nodes in categories tree by clicking on the arrow next to the category in the tree 4.Watch the logs and network logs

Redis enabled

Admin

Manage Categories

Open some child-nodes in categories tree

logs

Please refer the attached screenshots and let us know if we missed anything.

Add categories until you have a huge category tree with many subordinate categories

You need a lot of categories to reproduce the issue, not just 4. See also comment of @hostep

Add categories until you have a huge category tree with many subordinate categories

You need a lot of categories to reproduce the issue, not just 4. See also comment of @hostep

I have around 2700 categories in tree. This happens

I'm having the same issue, same error message. Strangely it is not being consistent along all our environments, even with all of them having the same Redis configuration.

The issue is not the Redis configuration, the issue is those hundreds (depending on the amount of categories) of concurrent calls to that categoriesJson url, @kandy I hope you understand it better by now?

Also, seeing this issue. As usual, Magento fails to test at scale. This is probably the 20th time I've seen an issue triggered when there are thousands of records and Magento says "We tested with 2 records and it's fine".

Why shouldn't everyone here just move to OroCommerce, BigCommerce, Shopify, Webflow, or one of the other platforms? Magento2 has drastically reduced in quality over the last 5 years.

Hi @netzkollektiv ,

Verified the issue on 2.4.7 again.

Preconditions

1.Redis enabled

Steps to reproduce:-

1.Go to the backend 2.Navigate to Manage Categories 3.Add categories until you have a huge category tree with many subordinate categories 4.Open some child-nodes in categories tree by clicking on the arrow next to the category in the tree 5.Watch the logs and network logs

Actual Result:

We are getting "Redis session exceeded concurrent connections" error and we can see many XHR requests coming to PageBuilder previews.

admin panel