Scaling admin-ajax.php.

[octalmage]

I work with a bunch of high traffic websites, and these websites are cached, hard. They want to use admin-ajax (or the Heartbeat API) to talk to the backend after a page has loaded but these requests have to bypass cache to get to the database. This can cause issues during increased traffic and our options are to disable those admin-ajax hits so all hits can be cached, or add additional hardware.

Do you have any tips for scaling ajax requests to WordPress? I've used Node.js for this personally (as a middleman) but I'd love to hear what your (and anyone elses!) experiences have been!

[markjaquith]

These requests don't HAVE to bypass the cache. If they are read-only requests, you could cache them for a reasonable amount of time. How you do this depends on the caching method you're using. But say you're using Varnish and you have a rule to return(pass); for anything in /wp-admin/. You could add to that check specific lookups for GET requests to /wp-admin/admin-ajax.php?action=something-that-can-be-cached. For instance, let's say you have a thing that does AJAX polling looking for new posts on the front page. You could cache this for a minute, so that the server only has to do one dynamic request per minute. Or, you could cache it for a longer time, and do proactive cache purging on that URL when something happens that you know invalidates its response (a new post being published, for example).

Another way to do this would be to use something like socket.io attached to a shared Redis backend to directly stream events to listeners. So, move away from the "poll, poll, poll" paradigm to an always-connected pub/sub paradigm.

[markjaquith]

Actually, I'm slightly amending my advice. admin-ajax.php sends nocache_headers() by default. So any upstream caching engine should NOT skip caching for all /wp-admin/admin-ajax.php requests, and instead rely on the cache headers sent by it to determine cache-ability. So, let's say you have an AJAX callback you want to be cacheable. First, you're going to have to "neuter" nocache_headers() for that request. Thankfully, this is WordPress, and there's a filter. nocache_headers.

function my_cache_friendly_ajax_call_headers( $headers ) {
    if ( defined( 'DOING_AJAX' ) && isset( $_GET['action'] ) && 'my-cache-friendly-action' === $_GET['action'] ) {
        $headers = array(); // Empty them
    }
    return $headers;
}

add_filter( 'nocache_headers', 'my_cache_friendly_ajax_call_headers', 9999 );

So we're checking to see if it’s a request for our special AJAX call, and wiping out the nocache headers if it is. Thus anything listening to the response upstream can cache it.

[markjaquith]

Caveats:

1. Make sure your conditionals for wiping out these headers are solid. Stuff is gonna seriously break if you're neglecting to set nocache headers in situations where you should.
2. If your page caching engine has a specific check for /wp-admin/ that always skips caching, you will need to modify it to exclude /wp-admin/admin-ajax.php.
3. If the response varies if the user is logged in, then check for that before wiping out the nocache headers. If you don't think it varies, or you don't want it to, then in your wp_ajax_my-cache-friendly-action action hook callback, do wp_set_current_user( 0 ); at the top, to make the rest of the functions you call act as if an anonymous user had done them.

[octalmage]

Thanks a ton Mark, this is really great stuff! The nocache_headers method is something our customers could easily implement.

Thanks again!