lukechampine / us

An alternative interface to Sia
MIT License
55 stars 4 forks source link

Various mux improvements #146

Closed lukechampine closed 3 years ago

lukechampine commented 3 years ago

Exported errors, more documentation, and better Write performance with 1000s of streams.

Current benchmarks:

BenchmarkConn               15664 ns/op         915.08 MB/s            0 B/op          0 allocs/op

BenchmarkRHP2               21776 ns/op         658.24 MB/s        14482 B/op          4 allocs/op

BenchmarkMux/1              16842 ns/op         851.08 MB/s            0 B/op          0 allocs/op
BenchmarkMux/2              37007 ns/op         774.67 MB/s            0 B/op          0 allocs/op
BenchmarkMux/10            177423 ns/op         807.90 MB/s            7 B/op          0 allocs/op
BenchmarkMux/100          1760469 ns/op         814.21 MB/s          221 B/op          0 allocs/op
BenchmarkMux/500          9458982 ns/op         757.69 MB/s         2814 B/op         22 allocs/op
BenchmarkMux/1000        18073882 ns/op         793.08 MB/s        10215 B/op         84 allocs/op

BenchmarkSiaMux/1           68773 ns/op         208.43 MB/s        87335 B/op         50 allocs/op
BenchmarkSiaMux/2          136385 ns/op         210.20 MB/s       174474 B/op         97 allocs/op
BenchmarkSiaMux/10         704057 ns/op         203.59 MB/s       871841 B/op        480 allocs/op
BenchmarkSiaMux/100       7029083 ns/op         203.92 MB/s      8728755 B/op       4785 allocs/op
BenchmarkSiaMux/500      35327472 ns/op         202.87 MB/s     43966534 B/op      24779 allocs/op
BenchmarkSiaMux/1000     71633701 ns/op         200.10 MB/s     88782438 B/op      51094 allocs/op

I also measured some latency quantiles; this is "time to Write ~14KiB:"

                           1%         10%         50%           90%        99%
BenchmarkMux/1           6.382µs   11.327µs    33.541µs     61.861µs     68.233µs
BenchmarkMux/2          17.698µs   21.024µs    35.807µs     50.591µs     53.917µs
BenchmarkMux/10        134.394µs   150.35µs   160.259µs    193.652µs    355.255µs
BenchmarkMux/100        15.626µs   280.56µs  1.458045ms   2.635531ms   2.900465ms
BenchmarkMux/500       105.497µs 1.155979ms   6.03254ms  12.819939ms  14.565243ms
BenchmarkMux/1000      113.842µs 2.657014ms 15.162247ms  31.571692ms  36.099315ms

                           1%         10%         50%           90%        99%
BenchmarkSiaMux/1       26.507µs   28.604µs    36.245µs    135.416µs    352.701µs
BenchmarkSiaMux/2       28.572µs   33.507µs    50.256µs    385.124µs    768.357µs
BenchmarkSiaMux/10      31.662µs   45.742µs   509.802µs   1.399073ms   2.449935ms
BenchmarkSiaMux/100     32.176µs   43.799µs  4.441839ms   15.54882ms  28.536804ms
BenchmarkSiaMux/500     35.866µs   48.586µs 20.105927ms  61.639938ms 137.245416ms
BenchmarkSiaMux/1000    37.623µs   50.332µs 42.052932ms 138.548955ms 284.248683ms
lukechampine commented 3 years ago

Goober, test and merge