search

Automattic / wp-cache-memcached

Memcached Object Cache for WordPress
6 stars 0 forks source link

Fatal error #3

Closed salvoaranzulla closed 1 year ago

salvoaranzulla commented 1 year ago

Hi,

I have tried your new object cache, but when I reload the page I see this error:

Fatal error: Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /path [...]/wp-cache-memcached/includes/wp-object-cache.php:851 Stack trace: #0 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(851): max(Array) #1 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache->get_max_flush_number('WP_Object_Cache') #2 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache->get_flush_number('WP_Object_Cache') #3 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache->get_blog_flush_number() #4 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(943): WP_Object_Cache->flush_prefix('default') #5 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache->key('is_blog_install...', 'default') #6 /path [...]/wp-cache-memcached/object-cache.php(172): WP_Object_Cache->get('is_blog_install...', '', false, NULL) #7 /path [..]/wp-includes/functions.php(1763): wp_cache_get('is_blog_install...') #8 /path [..]/wp-includes/load.php(791): is_blog_installed() #9 /path [..]/wp-settings.php(160): wp_not_installed() #10 /[path ..]/wp-config.php(139): require_once('/usr/share/ngin...') #11 /path [..]/wp-load.php(55): require_once('/usr/share/ngin...') #12 /path [..]/wp-blog-header.php(13): require_once('/usr/share/ngin...') #13 /[path ..]/index.php(17): require('/usr/share/ngin...') #14 {main} thrown in /path [...]/wp-cache-memcached/includes/wp-object-cache.php on line 851

Fatal error: Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /path [...]/wp-cache-memcached/includes/wp-object-cache.php:851 Stack trace: #0 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(851): max(Array) #1 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache->get_max_flush_number('WP_Object_Cache') #2 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache->get_flush_number('WP_Object_Cache') #3 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache->get_blog_flush_number() #4 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(943): WP_Object_Cache->flush_prefix('options') #5 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache->key('notoptions', 'options') #6 /path [...]/wp-cache-memcached/object-cache.php(172): WP_Object_Cache->get('notoptions', 'options', false, NULL) #7 /path [..]/wp-includes/option.php(165): wp_cache_get('notoptions', 'options') #8 /path [..]/wp-includes/l10n.php(63): get_option('WPLANG') #9 /path [..]/wp-includes/l10n.php(139): get_locale() #10 /path [..]/wp-includes/l10n.php(868): determine_locale() #11 /path [..]/wp-includes/class-wp-fatal-error-handler.php(47): load_default_textdomain() #12 [internal function]: WP_Fatal_Error_Handler->handle() #13 {main} thrown in /path [...]/wp-cache-memcached/includes/wp-object-cache.php on line 851

@WPprodigy Is there a way to fix it?

Thanks, Salvatore

WPprodigy commented 1 year ago

Hello!

So you'll need a global variable set up in the wp-config that looks similar to this:

$memcached_servers = array(
    'default' => array(
        '10.10.10.20:11211',
        '10.10.10.30:11211'
    )
);

Mapping to the address & port of the memcached instance(s).

More info is available in this readme: https://github.com/Automattic/wp-memcached

salvoaranzulla commented 1 year ago

Hi @WPprodigy,

Thanks for your reply.

1) I confirm that I have on my wp-config.php:

define('WP_CACHE', true);
define('AUTOMATTIC_MEMCACHED_USE_MEMCACHED_EXTENSION', true);
$memcached_servers = array(
    'default' => array(
        'elasticache-.....amazonaws.com:11211'
    )
);

2) The problem is strange. As you can see on first 10 refresh, the problem doesn't appear. After that, It starts to appear. I have recorded a video about it:

https://drive.google.com/file/d/1m_76fqqYacokD0G2U8_aKhvTuNUpWSx7/view?usp=sharing

Fatal error: Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /path [...]/wp-cache-memcached/includes/wp-object-cache.php:851 Stack trace: #0 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(851): max(Array) #1 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache->get_max_flush_number('WP_Object_Cache') #2 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache->get_flush_number('WP_Object_Cache') #3 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache->get_blog_flush_number() #4 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(943): WP_Object_Cache->flush_prefix('default') #5 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache->key('is_blog_install...', 'default') #6 /path [...]/wp-cache-memcached/object-cache.php(172): WP_Object_Cache->get('is_blog_install...', '', false, NULL) #7 /path [..]/wp-includes/functions.php(1763): wp_cache_get('is_blog_install...') #8 /path [..]/wp-includes/load.php(791): is_blog_installed() #9 /path [..]/wp-settings.php(160): wp_not_installed() #10 /[path ..]/wp-config.php(139): require_once('/usr/share/ngin...') #11 /path [..]/wp-load.php(55): require_once('/usr/share/ngin...') #12 /path [..]/wp-blog-header.php(13): require_once('/usr/share/ngin...') #13 /[path ..]/index.php(17): require('/usr/share/ngin...') #14 {main} thrown in /path [...]/wp-cache-memcached/includes/wp-object-cache.php on line 851

Fatal error: Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /path [...]/wp-cache-memcached/includes/wp-object-cache.php:851 Stack trace: #0 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(851): max(Array) #1 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache->get_max_flush_number('WP_Object_Cache') #2 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache->get_flush_number('WP_Object_Cache') #3 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache->get_blog_flush_number() #4 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(943): WP_Object_Cache->flush_prefix('options') #5 /path [...]/wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache->key('notoptions', 'options') #6 /path [...]/wp-cache-memcached/object-cache.php(172): WP_Object_Cache->get('notoptions', 'options', false, NULL) #7 /path [..]/wp-includes/option.php(165): wp_cache_get('notoptions', 'options') #8 /path [..]/wp-includes/l10n.php(63): get_option('WPLANG') #9 /path [..]/wp-includes/l10n.php(139): get_locale() #10 /path [..]/wp-includes/l10n.php(868): determine_locale() #11 /path [..]/wp-includes/class-wp-fatal-error-handler.php(47): load_default_textdomain() #12 [internal function]: WP_Fatal_Error_Handler->handle() #13 {main} thrown in /path [...]/wp-cache-memcached/includes/wp-object-cache.php on line 851

If I use the old https://wordpress.org/plugins/memcached-redux/, all works fine, but that drop-in plugin is very old.

Do you have any idea how to solve the problem?

Thanks, Salvatore

WPprodigy commented 1 year ago

A) Persistent Connection Issues?

Hmm, it sounds like the issue could be with the persistent connections perhaps. Maybe after ~10 requests the php-fpm workers are flushing or something but it's not being reset for some reason.

Because even if there were just one server and it had no flush number key in it, essentially this would happen:

wp> $values = max( array_map( 'intval', [ false ] ) );
=> int(0)

But if there are no connections in the default pool, then the array would be empty rather than containing false and thus the fatal.

Here is where the connection pool is conditionally setup in the newer adapter: https://github.com/Automattic/wp-cache-memcached/blob/develop/includes/memcached-adapter.php#L332-L363

When this fatal happens, it'd be interesting to know what these two functions output:

wp> $GLOBALS['wp_object_cache']->mc['default']->getServerList();

=> array(1) {
  [0]=>
  array(3) {
    ["host"]=>
    string(65) "10.10.10.20"
    ["port"]=>
    int(11211)
    ["type"]=>
    string(3) "TCP"
  }
}

$GLOBALS['wp_object_cache']->mc['default']->getServerByKey('test');

=> array(3) {
  ["host"]=>
  string(65) "10.10.10.20"
  ["port"]=>
  int(11211)
  ["weight"]=>
  int(0)
}

Also would also be interesting to check the result of this at the same time, as it'll show the error code the memcached server gave on the last request to it:

$GLOBALS['wp_object_cache']->mc['default']->getResultCode();

B) Memcached Server / Configuration Issues

Is it possible the Memcached server is dropping connections or reaching a maximum? I did some strenuous testing of the new persistent connections logic with large amounts of requests and it handled them fine, but is possible there are some configuration differences at play here, perhaps it's unable to (re)connect for some reason (which hopefully the getResultCode() could help point us towards).

C) Possible Quick Fix (else just kicking the bucket to the next problem)

A possible quick fix could be to do the following right here: https://github.com/Automattic/wp-cache-memcached/blob/develop/includes/wp-object-cache.php#L851

$max = empty( $values ) ? 0 : max( $values );

But I'm worried if there are no connections in the pool then you'll essentially just be moving forward with no object cache.

Another thing to potentially test is set if ( true ) { here: https://github.com/Automattic/wp-cache-memcached/blob/develop/includes/memcached-adapter.php#LL350C24-L350C24, effectively disabling persistent connections. But note that long-term and in production this is likely not ideal, and will inflate the total number of concurrent connections to the mc server (as well as just slow things down a bit since each will need to establish that initial TCP connection again)

salvoaranzulla commented 1 year ago

Hi @WPprodigy,

Thanks for your help.

1) I have put these lines:

        print_r($GLOBALS['wp_object_cache']->mc['default']->getServerList());
        print_r("\n\n");
        print_r($GLOBALS['wp_object_cache']->mc['default']->getServerByKey('test'));
        print_r("\n\n");
        print_r($GLOBALS['wp_object_cache']->mc['default']->getResultCode());
        print_r("\n\n");

in wp-object-cache.php before the line:

$max = max( $values );

When the object cache works fine, I see:


Array
(
    [0] => Array
        (
            [host] => elasticache-.......cache.amazonaws.com
            [port] => 11211
            [type] => TCP
        )

)

Array
(
    [host] => elasticache-.......cache.amazonaws.com
    [port] => 11211
    [weight] => 0
)

0

Array
(
    [0] => Array
        (
            [host] => elasticache-.......cache.amazonaws.com
            [port] => 11211
            [type] => TCP
        )

)

Array
(
    [host] => elasticache-.......cache.amazonaws.com
    [port] => 11211
    [weight] => 0
)

0

When the object cache stops to work, I see:


  | Array
-- | --
  | (
  | )
  |  
  |  
  |  
  |  
  | 20
  |  
  | <br />
  | <b>Fatal error</b>:  Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php:860
  | Stack trace:
  | #0 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(860): max(Array)
  | #1 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache-&gt;get_max_flush_number('WP_Object_Cache')
  | #2 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache-&gt;get_flush_number('WP_Object_Cache')
  | #3 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache-&gt;get_blog_flush_number()
  | #4 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(952): WP_Object_Cache-&gt;flush_prefix('default')
  | #5 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache-&gt;key('is_blog_install...', 'default')
  | #6 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/object-cache.php(172): WP_Object_Cache-&gt;get('is_blog_install...', '', false, NULL)
  | #7 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/functions.php(1763): wp_cache_get('is_blog_install...')
  | #8 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/load.php(791): is_blog_installed()
  | #9 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-settings.php(160): wp_not_installed()
  | #10 /usr/share/nginx/aranzulla.it/doc_root_tb/wp-config.php(140): require_once('/usr/share/ngin...')
  | #11 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-load.php(55): require_once('/usr/share/ngin...')
  | #12 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-blog-header.php(13): require_once('/usr/share/ngin...')
  | #13 /usr/share/nginx/aranzulla.it/doc_root_tb/index.php(17): require('/usr/share/ngin...')
  | #14 {main}
  | thrown in <b>/usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php</b> on line <b>860</b><br />
  | Array
  | (
  | )
  |  
  |  
  |  
  |  
  | 20
  |  
  | <br />
  | <b>Fatal error</b>:  Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php:860
  | Stack trace:
  | #0 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(860): max(Array)
  | #1 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache-&gt;get_max_flush_number('WP_Object_Cache')
  | #2 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache-&gt;get_flush_number('WP_Object_Cache')
  | #3 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache-&gt;get_blog_flush_number()
  | #4 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(952): WP_Object_Cache-&gt;flush_prefix('options')
  | #5 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache-&gt;key('notoptions', 'options')
  | #6 /usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/object-cache.php(172): WP_Object_Cache-&gt;get('notoptions', 'options', false, NULL)
  | #7 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/option.php(165): wp_cache_get('notoptions', 'options')
  | #8 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/l10n.php(63): get_option('WPLANG')
  | #9 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/l10n.php(139): get_locale()
  | #10 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/l10n.php(868): determine_locale()
  | #11 /usr/share/nginx/aranzulla.it/doc_root_tb/app/wp-includes/class-wp-fatal-error-handler.php(47): load_default_textdomain()
  | #12 [internal function]: WP_Fatal_Error_Handler-&gt;handle()
  | #13 {main}
  | thrown in <b>/usr/share/nginx/aranzulla.it/doc_root_tb/web/wp-cache-memcached/includes/wp-object-cache.php</b> on line <b>860</b><br />
  |  

If it could be useful, there is the PHP error log:

[09-May-2023 05:20:35 UTC] PHP Fatal error:  Uncaught ValueError: max(): Argument #1 ($value) must contain at least one element in /.../wp-cache-memcached/includes/wp-object-cache.php:860
Stack trace:
#0 /.../wp-cache-memcached/includes/wp-object-cache.php(860): max(Array)
#1 /.../wp-cache-memcached/includes/wp-object-cache.php(790): WP_Object_Cache->get_max_flush_number('WP_Object_Cache')
#2 /.../wp-cache-memcached/includes/wp-object-cache.php(777): WP_Object_Cache->get_flush_number('WP_Object_Cache')
#3 /.../wp-cache-memcached/includes/wp-object-cache.php(751): WP_Object_Cache->get_blog_flush_number()
#4 /.../wp-cache-memcached/includes/wp-object-cache.php(952): WP_Object_Cache->flush_prefix('default')
#5 /.../wp-cache-memcached/includes/wp-object-cache.php(339): WP_Object_Cache->key('is_blog_install...', 'default')
#6 /.../wp-cache-memcached/object-cache.php(172): WP_Object_Cache->get('is_blog_install...', '', false, NULL)
#7 /.../wp-includes/functions.php(1763): wp_cache_get('is_blog_install...')
#8 /.../wp-includes/load.php(791): is_blog_installed()
#9 /.../wp-settings.php(160): wp_not_installed()
#10 /.../wp-config.php(140): require_once('/usr/share/ngin...')
#11 /.../wp-load.php(55): require_once('/usr/share/ngin...')
#12 /.../wp-blog-header.php(13): require_once('/usr/share/ngin...')
#13 /.../index.php(17): require('/usr/share/ngin...')
#14 {main}
  thrown in /.../wp-cache-memcached/includes/wp-object-cache.php on line 860

2) We are using AWS ElasticCache with standard configuration. In the article https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/AutoDiscovery.Using.ModifyApp.PHP.html, AWS suggests using this configuration:

            \Memcached::OPT_CLIENT_MODE     => \Memcached::DYNAMIC_CLIENT_MODE,
            \Memcached::OPT_DISTRIBUTION    => \Memcached::DISTRIBUTION_CONSISTENT

I have tried to put this configuration on memcached-adapter.php but nothing is changed. So, If I put this configuration or I remove this configuration, the result is the same.

3) If I try:

$max = empty( $values ) ? 0 : max( $values );

I see a 502 Bad Gateway error when the object cache stops to work.

Do you have any suggestions?

Thanks, Salvatore

salvoaranzulla commented 1 year ago

@WPprodigy Another information, if it can help you.

4) We use AWS ElasticCache. In my wp-config.php I put the address of the cluster endpoints:

$memcached_servers = array(
    'default' => array(
        'elasticache-..-cluster-micro.....cfg.euc1.cache.amazonaws.com:11211'
    )
);

The cluster will return the endpoint. For example, elasticache-XXXX-XXXX-it-cluster.lfxxwy.cfg.euc1.cache.amazonaws.com:11211 will return elasticache-XXXXX-XXXXX-it-cluster.lfxxwy.0001.euc1.cache.amazonaws.com:11211 (.0001 for the first endpoint, .0002 for the second endpoint, etc).

I have also tried to put directly the final endpoint in wp-config.php, but nothing is changed: the error is the same with the same frequency.

5) I have also created a debug page with only PHP:

<?php

  $server_endpoint = "elasticache-xxxxx-xxxxx-cluster.lfxxwy.cfg.euc1.cache.amazonaws.com";
  $server_port = 11211;

  $dynamic_client = new Memcached('persistent-id');
  $dynamic_client->setOption(Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
  $dynamic_client->addServer($server_endpoint, $server_port);

  print_r($dynamic_client->getServerList());
  print_r($dynamic_client->getServerByKey('test'));
  print_r($dynamic_client->getResultCode());

I have done a lot of refreshes, the page always works fine:

Array ( [0] => Array ( [host] => elasticache-xxxxxx-cluster.lfxxwy.0001.euc1.cache.amazonaws.com [port] => 11211 [type] => TCP ) ) Array ( [host] => elasticache-xxxxxx-cluster.lfxxwy.0001.euc1.cache.amazonaws.com [port] => 11211 [weight] => 0 ) 0

So, I think that the bug is somewhere in the drop-in plugin.

WPprodigy commented 1 year ago

Interesting, so yeah there's definitely some sort of issue with server/connection persistence somehow. The error code you get for the last MC request is 20 / MEMCACHED_NO_SERVERS. And of course, the empty arrays also show that.

In your php-only script, I found that it's technically not using persistent connections because it gets reset whenever setOption/addServer is called. See https://www.php.net/manual/en/memcached.construct.php#93536

So if you try this, I imagine you might end up with a similar issue somehow:

$server_endpoint = "elasticache-xxxxx-xxxxx-cluster.lfxxwy.cfg.euc1.cache.amazonaws.com";
$server_port = 11211;

$dynamic_client = new Memcached('persistent-id');

if ( empty($dynamic_client->getServerList()) ) {
    $dynamic_client->setOption(Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
    $dynamic_client->addServer($server_endpoint, $server_port);
}

print_r($dynamic_client->getServerList());
print_r($dynamic_client->getServerByKey('test'));
print_r($dynamic_client->getResultCode());

If it does work, then I'd start to implement more of the logic seen here until it breaks again: https://github.com/Automattic/wp-cache-memcached/blob/develop/includes/memcached-adapter.php#L340-L350. That should help pinpoint the bug if there is one.

What does your general stack look like? Nginx with php-fpm, or something else? Asking because it's starting to sound like there may be something funky going on with the internal php memory.

Or might just be the DYNAMIC_CLIENT_MODE thing, of which I'm not finding much info about online outside of AWS: https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/AutoDiscovery.Using.ModifyApp.PHP.html

By configuring the client with the Dynamic client mode with single endpoint, the client will periodically use the configuration endpoint to retrieve the current cache cluster configuration. This allows scaling the cache cluster up or down in number of nodes without requiring any changes to the PHP application.

I imagine this is where things aren't going well. On our infrastructure we have some other configuration reloading in place that automatically updates the $memcached_servers global when needed. But in your case if AWS is either changing the config info or some sort of "internal endpoint" is trying to refresh it, that might not being playing nice with the logic linked above.

Maybe try out Memcached::STATIC_CLIENT_MODE?

WPprodigy commented 1 year ago

Also though, going off of https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/AutoDiscovery.Using.ModifyApp.PHP.html, if you want to use their auto discovery feature set then I think it's just a matter of stripping out most of this part: https://github.com/Automattic/wp-cache-memcached/blob/develop/includes/memcached-adapter.php#L337-L358

And just init-ing things with the recommended approach AWS gives. Their example is questionable on if it's really persistent or not though.

salvoaranzulla commented 1 year ago

Hi @WPprodigy,

Thanks for your help and your patience.

1) I have tried your PHP script and after few refreshes I see an empty array. Than, I have changed Memcached::DYNAMIC_CLIENT_MODE -----> Memcached::STATIC_CLIENT_MODE and all works fine.

2) I have added the following line on memcached-adapter.php and all works fine:

\Memcached::OPT_CLIENT_MODE => \Memcached::STATIC_CLIENT_MODE,

3) With AWS ElasticSearch, you can configure a Memcached cluster and you get the cluster endpoint. If I add a new server on the cluster, the DYNAMIC_CLIENT_MODE will auto-discover them.

So I think, as you said, that we need to use \Memcached::STATIC_CLIENT_MODE or just removing https://github.com/Automattic/wp-cache-memcached/blob/develop/includes/memcached-adapter.php#L337-L358 to allow the DYNAMIC_CLIENT_MODE.

What do you think?

salvoaranzulla commented 1 year ago

@WPprodigy

4) I have also found this: https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/AutoDiscovery.HowAutoDiscoveryWorks.html

5) I confirm your intuition. If we want to use che "configuration endpoint" (cluster endpoint), I need to use Memcached::DYNAMIC_CLIENT_MODE.

If I copy the address of each nodes in wp-config.php and all works fine (Memcached::STATIC_CLIENT_MODE is not needed): I think that I will use this way, copying the address of each nodes. If in the future, I will create new nodes, I will put manually in wp-config.php. What do you think about this?

WPprodigy commented 1 year ago

Cool, glad we got to the bottom of this, and I learned something new :). Although now I'm pretty curious as to how AWS has built-in these new flags into the Memcached extension and how it works. Will have to check that out some time!

Copying the addresses of the nodes directly into $memcached_servers would be closest to what this plugin was developed for. But I should note that we do have automated logic that updates $memcached_servers for us whenever the topology changes. And I'm not sure how often you might expect AWS to be changing the nodes around (node crashes/updates/etc).

Personally I would probably try to stick with the recommended approach from AWS w/ the DYNAMIC_CLIENT_MODE. However it does seem to maybe be a bit buggy when attempting to use truly persistent connections - so I'm not sure how that will work out. Would need to set up some metrics that keeps track of both active connections and the connection rate to see if their modifications to the extension do allow for persistence whilst still calling addServer() on every request. Ultimately it may not matter too much though.

salvoaranzulla commented 1 year ago

Hi @WPprodigy,

Thanks for your help. I confirm that if we use the address of each node, all works fine. I think I will use this strategy because we don't do any kind of autoscaling on Elasticache and We use the same nodes since 4 years.