littlebizzy / slickstack

Lightning-fast WordPress on Nginx
https://slickstack.io
GNU General Public License v3.0
638 stars 112 forks source link

Add a sysctl for UDS max connections #15

Closed zardoz-bzar closed 4 years ago

zardoz-bzar commented 5 years ago

The default for a UDS is 128 connections.

After 128 concurrent connections between nginx and php-fpm, you start getting errors.

(replicate with using apachebench, ab -n 10000 -c 150 localhost/some_simple.php)

I suggest setting the net.core.somaxconn to 4096 as a minimum with the current nginx config, or 65535.

I think conventional wisdom is that at high concurrency, you really want to be using loopback instead of the domain socket for php-fpm at the level you are setting nginx at.

Appreciate all the work you've done combining all the various nginx settings together and citing the sources!

-zardoz

jessuppi commented 5 years ago

Thanks, please review if interested and let us know if any other suggestions. We will add the kernel config to the ss-install script after tests appear stable:

https://github.com/littlebizzy/slickstack/blob/master/ubuntu/sysctl.txt

Also for reference:

https://github.com/littlebizzy/slickstack/blob/master/nginx/nginx-conf.txt

zardoz-bzar commented 5 years ago

On first glance it looks good. Removes most of what I call, "the rakes in the yard that cause 502 errors".

Might need a note attached saying, "This is so that you don't run into common errors that force you to pay someone a consulting fee for a few simple tunables. Be aware that nginx, php, and your database are all competing for the same resources if you run them on the same host. These settings should work fine behind Cloudflare, and you should be using it since it's free. If you aren't using Cloudflare, you can adjust these down as necessary or move the DB to a different host."

So... awesome job. very thorough.

People spend a lot of time trying to use plugins with CF, but it I think these two rules protect the main site with all the tunables maxed, but offload the heaviest lifting (images) entirely. example.com/wp-content/uploads/ Cache-Everything example.com/ Cache-Standared Origin Cache Control

Nginx works as a second level cache so you don't need the bypass-on-cookie 20 buck tax. Woocomm even works with those rules and with or without a second level nginx cache in place, so they're pretty safe. Maybe you have a better rule recommendation for the free plan too as I haven't tested those thoroughly yet beyond some ecom sites (and those sites never double-sell items, have cart issues, etc).

You've done a good job democratizing the knowledge.

I might have one of my interns actually draw a diagram for this and submit it you. :-)

jessuppi commented 5 years ago

I suggest setting the net.core.somaxconn to 4096 as a minimum with the current nginx config, or 65535.

net.core.somaxconn and tons of other rules now optimized in our latest Ubuntu kernel, and I've gone ahead and made this part of the ss-install process now:

https://github.com/littlebizzy/slickstack/blob/master/ubuntu/sysctl.txt

Previously, the default kernel was not modified.


I think conventional wisdom is that at high concurrency, you really want to be using loopback instead of the domain socket for php-fpm at the level you are setting nginx at.

From my understanding, Unix sockets are better due to acting as a "shortcut" in the stack instead of having to go through various networking layers, etc:

https://unix.stackexchange.com/questions/91774/performance-of-unix-sockets-vs-tcp-ports

https://stackoverflow.com/questions/257433/postgresql-unix-domain-sockets-vs-tcp-sockets/257479

https://stackoverflow.com/questions/42704763/what-are-the-differences-from-running-php-fpm-over-an-unix-socket-vs-a-tcp-ip-so

Anyway let's keep this topic focus on the net.core kernel rules if possible. Please kindly create new Issues if you can expand more on your other suggestions, thanks! 🤓

jessuppi commented 5 years ago

Okay I understand now why you lumped some of these topics together. After further research I've changed the Nginx and PHP-FPM config to 127.0.0.1 (TCP/IP) instead of sockets:

https://github.com/littlebizzy/slickstack/blob/master/php-fpm/www-conf.txt https://github.com/littlebizzy/slickstack/blob/master/nginx/server-block-single-site.txt

It seems Unix sockets are technically faster, but with two drawbacks:

  1. More disk I/O over the long-term
  2. At extremely high traffic, the risk of socket errors

The security concerns regarding TCP/IP are largely mitigated with listen.allowed_clients