Added Caddyserver to TechEmpower benchmark

[joanhey]

Please review/optimize this PR https://github.com/TechEmpower/FrameworkBenchmarks/pull/7689

Thank you

[mholt]

I would recommend adding https://frankenphp.dev/ -- it is also written with Caddy and performs much better than FastCGI!

[joanhey]

I have started frankephp, but first we need to add Caddyserver. Perhaps the next week, I'll push frankenphp.

In reality, I made some bets frankephp vs workerman, than workerman is faster.

[joanhey]

Now we will test: caddy, h2o, and nginx, all using php-fpm.

Later frankenphp vs workerman and ngx-php.

[joanhey]

Please don't see the benchmark as a competition.

A benchmark is a very good tool to optimize the performance of the apps. Thank you

[mholt]

Thanks for keeping us informed :)

(Closing, as nothing actionable for us to do.)

[joanhey]

https://www.techempower.com/benchmarks/#section=test&runid=03363299-b1b5-42e6-8ca5-6e0665be82c2&test=fortune&f=zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-yelngf-zik0zj-zik0zj-zik0zj-zik0zj-zik0zj-sf

Nginx Unit failed in the last 2 runs, but have +100K req/s

When finish the run, I'll send you the link to the logs.

[joanhey]

Latency

[francislavoie]

That latency one makes no sense. There's clearly a problem there.

[joanhey]

php, php-h2o and php-caddy use the same php-fpm config file. And the same php files.

[francislavoie]

I can guarantee that Caddy itself does not introduce that much latency (especially for something on the same machine). There's clearly a problem with the test setup.

[joanhey]

Clearly is NOT a problem with the bench. Like I said before, a benchmark is a very good tool to check anomalies.

[francislavoie]

Benchmarks are not immune to flaws.

32ms latency on something on the same machine makes no sense at all.

Caddy setups with PHP-FPM in Docker that I have running in production respond in sub-millisecond time.

[joanhey]

The code is also not immune. The bench it's working with all platforms, but only fail with Caddy ??

There are 3 servers for the bench:

• 1 with the server and app
• 1 with the database
• 1 that is the client with Wrk

Testing in the same machine, is a very bad synthetic bench, and never realistic.

[joanhey]

I have nothing against Caddyserver. For me will be perfect, to have another fast server option to use.

[joanhey]

What I don't like are Fake Benchs and Bench Marketing. I added Caddyserver after watch a fake bench about Caddy and Nginx.

I make benchmarks for more than 15 years, and normally for help the apps.

[joanhey]

Latency is dependent. The latency is relative to the concurrent users and req/s. With 100 req/s and 100 users, is low. With more req/s the latency go higher.

[mholt]

I added Caddyserver after watch a fake bench about Caddy and Nginx.

Where was that? I'd like to see.

Also, what version of Caddy are you using?

How can we reproduce your results?

[henrystivens]

Hi @mholt

The results are provide for Techempower benchmark, are open source:

https://www.techempower.com/benchmarks/#section=data-r21 [Round 21]

You can download from Github and run the tests:

https://github.com/TechEmpower/FrameworkBenchmarks

[henrystivens]

The docker file for Caddy and other servers you cand find in PHP folder:

https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/PHP/php

[joanhey]

The Fake bench: https://blog.tjll.net/reverse-proxy-hot-dog-eating-contest-caddy-vs-nginx/

I created issues, and PRs. https://github.com/tylerjl/benchmarking-proxies/issues https://github.com/tylerjl/benchmarking-proxies/pulls

But only appear the appendix from Caddy.

Ok I'll add to a neutral benchmark.

[joanhey]

I helped and help, a lot of fw and servers to fix anomalies. If you don't measure it, you can't know how to improve it.

And here was the worst welcome that I received never.

[joanhey]

Read the issues!!! https://github.com/tylerjl/benchmarking-proxies/issues

[joanhey]

What version am I using?

[mholt]

Ok, the rude and demanding tone in this thread, along with posting multiple repeated small comments, is not acceptable behavior, so I'm locking this.