read error on connection on crawlers search bot

[altitude-it]

Hi Colin,

I have intermittently the following error when bots crawls through my website. I using a magento EE 1.13.1.0 within a cluster of 7 web instances (nginx), 1 admin instance (apache), 3 redis instances (FPC, backend, session) and 1 db instances (mysql 5.6).

here the trace exception from one app instance :

Trace:

0 /www/data/lib/Credis/Client.php(603): Credis_Client->__call('select', Array)

1 /www/data/lib/Cm/Cache/Backend/Redis.php(135): Credis_Client->select(0)

2 /www/data/lib/Zend/Cache.php(153): Cm_Cache_Backend_Redis->__construct(Array)

3 /www/data/lib/Zend/Cache.php(94): Zend_Cache::_makeBackend('Cm_Cache_Backen...', Array, true, true)

4 /www/data/app/code/core/Mage/Core/Model/Cache.php(137): Zend_Cache::factory('Varien_Cache_Co...', 'Cm_Cache_Backen...', Array, Array, true, true, true)

5 /www/data/app/code/core/Mage/Core/Model/Config.php(1348): Mage_Core_Model_Cache->__construct(Array)

6 /www/data/app/Mage.php(463): Mage_Core_Model_Config->getModelInstance('core/cache', Array)

7 /www/data/app/code/core/Enterprise/PageCache/Model/Cache.php(68): Mage::getModel('core/cache', Array)

8 /www/data/app/code/local/Enterprise/PageCache/Model/Processor.php(195): Enterprise_PageCache_Model_Cache::getCacheInstance()

9 /www/data/app/code/local/Enterprise/PageCache/Model/Processor.php(157): Enterprise_PageCache_Model_Processor->_getDesignPackage()

10 /www/data/app/code/local/Enterprise/PageCache/Model/Processor.php(116): Enterprise_PageCache_Model_Processor->_createRequestIds()

11 /www/data/app/code/core/Mage/Core/Model/Cache.php(703): Enterprise_PageCache_Model_Processor->__construct()

12 /www/data/app/code/core/Mage/Core/Model/Cache.php(685): Mage_Core_Model_Cache->_getProcessor('Enterprise_Page...')

13 /www/data/app/code/local/Mage/Core/Model/App.php(340): Mage_Core_Model_Cache->processRequest()

14 /www/data/app/Mage.php(684): Mage_Core_Model_App->run(Array)

15 /data/web/altitude-sports.com/releases/20150422161736/index.php(153): Mage::run('mywebsite_...', 'website')

16 {main}

additionnally here's my magento setup :

    <session_save>db</session_save>
    <!--<session_cache_limiter><![CDATA[no-store, no-cache, must-revalidate, post-check=0, pre-check=0]]></session_cache_limiter>-->
    <redis_session>
        <host>XXXXXX</host>
        <port>6379</port>
        <password></password>
        <timeout>30</timeout>
        <persistent></persistent>
        <db>0</db>
        <compression_threshold>2048</compression_threshold>
        <compression_lib>lzf</compression_lib>
        <log_level>1</log_level>
        <log_broken_locks>0</log_broken_locks>
        <max_concurrency>32</max_concurrency>
        <break_after_frontend>5</break_after_frontend>
        <break_after_adminhtml>30</break_after_adminhtml>
        <first_lifetime>600</first_lifetime>
        <bot_lifetime>7200</bot_lifetime>
    </redis_session>
    <cache>
      <backend>Cm_Cache_Backend_Redis</backend>
      <backend_options>
        <server>XXXX</server>
        <port>6379</port>
        <persistent></persistent>
        <database>0</database>
        <password></password>
        <force_standalone>0</force_standalone>
        <connect_retries>1</connect_retries>
        <read_timeout>10</read_timeout>
        <log_level>4</log_level>
        <automatic_cleaning_factor>0</automatic_cleaning_factor>
        <compress_data>1</compress_data>
        <compress_tags>1</compress_tags>
        <compress_threshold>20480</compress_threshold>
        <compression_lib>lzf</compression_lib>
      </backend_options>
    </cache>
    <full_page_cache>
      <backend>Cm_Cache_Backend_Redis</backend>
      <backend_options>
        <server>XXXX</server>
        <port>6379</port>
        <persistent></persistent>
        <database>0</database>
        <password></password>
        <force_standalone>0</force_standalone>
        <connect_retries>1</connect_retries>
        <lifetimelimit>86400</lifetimelimit>
        <compress_data>0</compress_data>
      </backend_options>
    </full_page_cache>        

regards, jack

[colinmollenhour]

This seems not too uncommon for Redis and I don't know that there is a common solution. Things to try: disable/enable persistent connections, disable/enable force_standalone, increase read_timeout, add ping command after the connection is established, etc. If none of the above work, get ready to start Googling.. Lots of issues on github and StackOverflow questions on this topic.

I would be really interested to see what kind of traffic you are serving with that setup. E.g. fpm requests per second, Redis requests per second, etc.