search

colinmollenhour / Cm_Cache_Backend_Redis

A Zend_Cache backend for Redis with full support for tags (works great with Magento)
Other
389 stars 142 forks source link

protocol error, got 'A' as reply type byte #97

Closed csdougliss closed 8 years ago

csdougliss commented 8 years ago

Latest code, Redis is 3.0.4. Magento is EE 1.14.2.1.

a:4:{i:0;s:43:"protocol error, got 'A' as reply type byte
";i:1;s:1703:"#0 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Credis/Client.php(603): Credis_Client->__call('select', Array)
#1 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Cm/Cache/Backend/Redis.php(144): Credis_Client->select(0)
#2 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Zend/Cache.php(153): Cm_Cache_Backend_Redis->__construct(Array)
#3 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Zend/Cache.php(94): Zend_Cache::_makeBackend('Cm_Cache_Backen...', Array, true, true)
#4 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/Cache.php(137): Zend_Cache::factory('Varien_Cache_Co...', 'Cm_Cache_Backen...', Array, Array, true, true, true)
#5 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/Config.php(1348): Mage_Core_Model_Cache->__construct(Array)
#6 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/Mage.php(471): Mage_Core_Model_Config->getModelInstance('core/cache', Array)
#7 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/App.php(412): Mage::getModel('core/cache', Array)
#8 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/App.php(306): Mage_Core_Model_App->_initCache(Array)
#9 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/App.php(348): Mage_Core_Model_App->baseInit(Array)
#10 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/Mage.php(692): Mage_Core_Model_App->run(Array)
#11 /var/www/vhosts/magento-capistrano/releases/20150929123414/index.php(111): Mage::run('vax_yard_sale', 'store')
#12 {main}";s:3:"url";s:517:"/turpentine/esi/getBlock/method/esi/access/private/ttl/0/hmac/69759270a7cd3e8986bc679b0b5782409d4de29c7f7977fcb3c5c622c56f3727/data/JaAJVe-463lN.S7ABcBMOeNS5HnSl485gvCJhH7woq.OuG7GTSVxGTaladziX6XlwzKGjWKbHy3KhyAtWtTMOyyP3ZaDGWQRGHcdtWyYT1KD3rFpUXD1nByXwB0gIMAltkxrWUPDUXs3.ZBxOJSuGqfqAXKUnxI2jYb.CFphi.2.q0N7C0Y2Fm6Okfd0nMppQieAjX1YuMmQwaAlMQ-sNYujmrmh0V89Z7fIDncPxgOywziG9cX6gMlTMGO23R9IPU7sahzC1yxYkJ9VKqbvqrMPYM-ojF86E45idNTdFr3g6ZYZewtmHKzY4-M3mhUytE8hNAhKcgrDuTcV2RmNH9gQc6HYmBnSJBOYNrOyJO2nf84f54ihQQywBXGtpTHB/";s:11:"script_name";s:10:"/index.php";}root@web:/var/www/vhosts/magento-capistrano/shared/var/report
csdougliss commented 8 years ago

Under load seeing very similar:

a:4:{i:0;s:43:"protocol error, got '1' as reply type byte
";i:1;s:1703:"#0 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Credis/Client.php(603): Credis_Client->__call('select', Array)
#1 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Cm/Cache/Backend/Redis.php(144): Credis_Client->select(0)
#2 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Zend/Cache.php(153): Cm_Cache_Backend_Redis->__construct(Array)
#3 /var/www/vhosts/magento-capistrano/releases/20150929123414/lib/Zend/Cache.php(94): Zend_Cache::_makeBackend('Cm_Cache_Backen...', Array, true, true)
#4 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/Cache.php(137): Zend_Cache::factory('Varien_Cache_Co...', 'Cm_Cache_Backen...', Array, Array, true, true, true)
#5 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/Config.php(1348): Mage_Core_Model_Cache->__construct(Array)
#6 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/Mage.php(471): Mage_Core_Model_Config->getModelInstance('core/cache', Array)
#7 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/App.php(412): Mage::getModel('core/cache', Array)
#8 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/App.php(306): Mage_Core_Model_App->_initCache(Array)
#9 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/code/core/Mage/Core/Model/App.php(348): Mage_Core_Model_App->baseInit(Array)
#10 /var/www/vhosts/magento-capistrano/releases/20150929123414/app/Mage.php(692): Mage_Core_Model_App->run(Array)
#11 /var/www/vhosts/magento-capistrano/releases/20150929123414/index.php(111): Mage::run('vax_yard_sale', 'store')
#12 {main}";s:3:"url";s:693:"/turpentine/esi/getBlock/method/esi/access/private/ttl/14400/hmac/a182024e441422a10690c825010d57f20497cd6c1fb9d2f407cdc1b1418fa7d5/data/Mjn5-rPcA1BNj320-gHbFghUmxkrP3VEdbRVt1d96aVB1apX0VouDWYiCa2weU4An3ISqaUusImAj3J4qqDlklT.0DQ.V.axSNcGV9aiEDtbIDvmqj0wuUTGKbHsGzoBv.7BkHXIDOjaNO97WRu5Jkxd7B.ZEsxtW1uidBxZQB.mQQ2GgIzZo2pGolyCZB6mdxrr36wqhC-6rtO7lXSvgVgXK1Bl9Y5bm92eSLpEeAxCAGgpYA1yEeStoCHj..EHMHcyRXXj.YPzliljgNNC43S9v8-kZD2MrWbjJpDJTKrfEjbDaqSBbzG8Dp0MYjK.Ou4r5yhX-OI4MxrsC54RwwBD40-KgL1v4ifel6ZBaohPQt9OdzXHzvEcHG4oMVJgQrH4hUj9W9cJ51W3vg9U2LSQZVeHvchCB9pJxttudGEJG795XJpSS3-ba.QG6CPOZjO5uYigF4NYqW0727bt9xnGVt0oXW40tcF0r.VpVlKkeo2gS96bE0gVqOlw3xdnxcocrMzh1RfGK3fo8yPnaAhfqWQUvNI9ewBWetnymS4=/";s:11:"script_name";s:10:"/index.php";}root@web:/var/www/vhosts/magento-capistrano/shared/var/repor
WinstonN commented 8 years ago

I saw this today. My site fell over completely. It's like my network & cpu got maxed out

colinmollenhour commented 8 years ago

I'm curious, how much traffic did it take to produce this issue?

WinstonN commented 8 years ago

@colinmollenhour I had 8000 concurrents It looks like my redis maxed out on the cpu - at the peak the Network Out of the instance was 5 gbps

It pretty much started these errors from around 5000 concurents and lasted all the way through.

I have also seen other variants of this same error, for example, I saw protocol error, got 'f' as reply type byte

aoldoni commented 8 years ago

@WinstonN and @craigcarnell , I am also consistently seeing this error on heavy loads during load tests, similar amount of "concurrents".

Looks like there are several open issues in phpredis regarding this:

Were there any adjustments that helped your setup stop showing these errors?

Thanks.

colinmollenhour commented 8 years ago

I personally dropped phpredis project when I went to PHP7. I haven't done another test to see how performance is affected but going to PHP7 was such a big boost that even if standalone is slower PHP7 is so much faster that I don't worry about it. Anyway, this definitely is a phpredis bug and not a Cm_Cache_Backend_Redis bug.