perftest: transfering bytes, no encryption, pipes vs. socketpairs

[gperciva]

This is for investigating the variability in the existing standalone_pipe_socketpair_one.c test, which uses proto_pipe(). I took that test, replace the proto_pipe with a read(); write(); loop, and then made the communication endpoints either pipes, socketpairs, or socketpairs with the same socket options that are used in proto_conn.c.

Here's two visualizations of the same benchmarks, with N=31. Freebsd is 12.4 on bare metal; Linux is aws linux on a t2.

• the setsockopt() variant seems to be the same as the plain socketpairs, so I don't think it's worth including that as a separate test. (Unless I'm missing something some options that proto_conn.c used?)
• on FreeBSD, using pipes has much less variability.
• on Linux, socketpairs are surprisingly slower than pipes.

I didn't play games with cpu pinning, because pipes vs. sockets seemed to be an interesting enough comparison as it is.

So my proposal is to omit the setsockopt() test, and and only merge the first two commits in this PR.

[cperciva]

Please retest with linux on a t3 instance; t2 is Xen-based and may have performance anomalies if Linux uses page-flipping.

It would also be good to check FreeBSD 13.2 (not 12.4) and on a comparable EC2 instance. I'm guessing FreeBSD isn't really 2x faster than Linux.

[gperciva]

Interesting results! t3.small, 5 warmup runs, N=101 recorded data

• freebsd 13.2, with clang 14.0.5 (default cc)
• amazon linux 15.0.6-3.amzn2023.0.2, with clang 15.0.6 (sudo yum install clang)

Two visualizations of the same data:

My take-aways:

• yes, freebsd is twice as fast (with this test). And up to 3 times as fast with pipes.
• in freebsd 13.2, pipe() has the variability, while socket() doesn't. That's the opposite of my bare-metal 12.4 system (intel core i3-71000U).
• the setsockopt() stuff makes no difference, so there's no point committing it.

I didn't play any games with cpu pinning or sysctl kern.timecounter.alloweddeviation or the like.

[gperciva]

I think that the pipe() vs. socketpair() test is interesting enough to be worth adding to git master (as part of standalone-enc).