Latency is high

[simongui]

Throughput of Redcon seems really good but latency is really high in comparison to Redis with shallow or deep pipelines. Even if you make the Redis commands NOOP.

I created a SET command that only wrote a response to benchmark the I/O performance of redcon versus Redis. Keep in mind Redis is doing key mutations while my benchmark is not.

conn.WriteString("OK")

Redis

redis-benchmark -h server -p 6379 -t set -n 10000000 -P 1 -c 768
====== SET ======
  10000000 requests completed in 101.57 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
0.00% <= 3 milliseconds
76.97% <= 4 milliseconds
99.98% <= 5 milliseconds
99.99% <= 6 milliseconds
100.00% <= 7 milliseconds
100.00% <= 8 milliseconds
100.00% <= 8 milliseconds
98455.23 requests per second

redis-benchmark -h server -p 6379 -t set -n 10000000 -P 256 -c 768
====== SET ======
  10000000 requests completed in 8.61 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 8 milliseconds
0.00% <= 9 milliseconds
0.03% <= 10 milliseconds
0.06% <= 78 milliseconds
0.06% <= 79 milliseconds
0.10% <= 80 milliseconds
0.13% <= 81 milliseconds
0.14% <= 82 milliseconds
0.16% <= 84 milliseconds
0.17% <= 85 milliseconds
0.18% <= 86 milliseconds
0.19% <= 87 milliseconds
0.22% <= 88 milliseconds
0.27% <= 89 milliseconds
0.32% <= 90 milliseconds
0.35% <= 91 milliseconds
0.36% <= 92 milliseconds
0.36% <= 96 milliseconds
0.38% <= 97 milliseconds
0.39% <= 98 milliseconds
0.41% <= 99 milliseconds
0.43% <= 100 milliseconds
0.48% <= 101 milliseconds
0.55% <= 102 milliseconds
0.62% <= 103 milliseconds
0.68% <= 104 milliseconds
0.75% <= 105 milliseconds
0.83% <= 106 milliseconds
0.91% <= 107 milliseconds
0.98% <= 108 milliseconds
1.07% <= 109 milliseconds
1.16% <= 110 milliseconds
1.23% <= 111 milliseconds
1.29% <= 112 milliseconds
1.35% <= 113 milliseconds
1.42% <= 114 milliseconds
1.47% <= 115 milliseconds
1.54% <= 116 milliseconds
1.60% <= 117 milliseconds
1.66% <= 118 milliseconds
1.73% <= 119 milliseconds
1.80% <= 120 milliseconds
1.86% <= 121 milliseconds
1.89% <= 122 milliseconds
1.91% <= 128 milliseconds
1.94% <= 129 milliseconds
2.05% <= 130 milliseconds
2.24% <= 131 milliseconds
2.57% <= 132 milliseconds
3.32% <= 133 milliseconds
4.32% <= 134 milliseconds
5.45% <= 135 milliseconds
6.56% <= 136 milliseconds
7.74% <= 137 milliseconds
9.23% <= 138 milliseconds
10.92% <= 139 milliseconds
12.76% <= 140 milliseconds
14.70% <= 141 milliseconds
16.73% <= 142 milliseconds
18.77% <= 143 milliseconds
20.73% <= 144 milliseconds
22.69% <= 145 milliseconds
24.62% <= 146 milliseconds
26.42% <= 147 milliseconds
27.84% <= 148 milliseconds
29.06% <= 149 milliseconds
30.14% <= 150 milliseconds
30.95% <= 151 milliseconds
31.64% <= 152 milliseconds
32.47% <= 153 milliseconds
33.75% <= 154 milliseconds
35.64% <= 155 milliseconds
38.12% <= 156 milliseconds
41.20% <= 157 milliseconds
44.65% <= 158 milliseconds
48.27% <= 159 milliseconds
52.01% <= 160 milliseconds
55.82% <= 161 milliseconds
59.59% <= 162 milliseconds
63.34% <= 163 milliseconds
67.20% <= 164 milliseconds
71.02% <= 165 milliseconds
74.71% <= 166 milliseconds
77.64% <= 167 milliseconds
79.56% <= 168 milliseconds
81.13% <= 169 milliseconds
82.53% <= 170 milliseconds
83.86% <= 171 milliseconds
85.15% <= 172 milliseconds
86.44% <= 173 milliseconds
87.65% <= 174 milliseconds
88.88% <= 175 milliseconds
90.05% <= 176 milliseconds
91.20% <= 177 milliseconds
92.25% <= 178 milliseconds
93.18% <= 179 milliseconds
94.07% <= 180 milliseconds
94.89% <= 181 milliseconds
95.64% <= 182 milliseconds
96.34% <= 183 milliseconds
96.90% <= 184 milliseconds
97.33% <= 185 milliseconds
97.63% <= 186 milliseconds
97.90% <= 187 milliseconds
98.12% <= 188 milliseconds
98.36% <= 189 milliseconds
98.58% <= 190 milliseconds
98.79% <= 191 milliseconds
98.98% <= 192 milliseconds
99.16% <= 193 milliseconds
99.34% <= 194 milliseconds
99.48% <= 195 milliseconds
99.60% <= 196 milliseconds
99.71% <= 197 milliseconds
99.84% <= 198 milliseconds
99.91% <= 199 milliseconds
99.96% <= 200 milliseconds
99.99% <= 201 milliseconds
100.00% <= 201 milliseconds
1161170.38 requests per second

Redcon

redis-benchmark -h server -p 6380 -t set -n 10000000 -P 1 -c 768
====== SET ======
  10000000 requests completed in 90.21 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.01% <= 2 milliseconds
28.35% <= 3 milliseconds
81.28% <= 4 milliseconds
99.97% <= 5 milliseconds
99.98% <= 6 milliseconds
99.98% <= 260 milliseconds
99.98% <= 261 milliseconds
99.98% <= 262 milliseconds
99.98% <= 263 milliseconds
99.98% <= 264 milliseconds
99.98% <= 265 milliseconds
99.98% <= 266 milliseconds
99.98% <= 267 milliseconds
99.98% <= 268 milliseconds
99.98% <= 269 milliseconds
99.98% <= 270 milliseconds
99.99% <= 271 milliseconds
99.99% <= 272 milliseconds
99.99% <= 273 milliseconds
99.99% <= 274 milliseconds
99.99% <= 275 milliseconds
99.99% <= 864 milliseconds
99.99% <= 865 milliseconds
99.99% <= 866 milliseconds
99.99% <= 867 milliseconds
99.99% <= 868 milliseconds
100.00% <= 869 milliseconds
100.00% <= 870 milliseconds
100.00% <= 871 milliseconds
100.00% <= 872 milliseconds
100.00% <= 873 milliseconds
100.00% <= 873 milliseconds
110853.69 requests per second

redis-benchmark -h server -p 6380 -t set -n 10000000 -P 256 -c 768
====== SET ======
  10000000 requests completed in 4.47 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
0.00% <= 9 milliseconds
0.02% <= 10 milliseconds
0.06% <= 11 milliseconds
0.17% <= 12 milliseconds
0.29% <= 13 milliseconds
0.44% <= 14 milliseconds
1.33% <= 15 milliseconds
2.41% <= 16 milliseconds
3.83% <= 17 milliseconds
5.83% <= 18 milliseconds
8.42% <= 19 milliseconds
11.30% <= 20 milliseconds
14.10% <= 21 milliseconds
16.94% <= 22 milliseconds
19.60% <= 23 milliseconds
22.31% <= 24 milliseconds
25.01% <= 25 milliseconds
27.65% <= 26 milliseconds
30.40% <= 27 milliseconds
33.02% <= 28 milliseconds
35.74% <= 29 milliseconds
38.77% <= 30 milliseconds
41.45% <= 31 milliseconds
44.18% <= 32 milliseconds
46.89% <= 33 milliseconds
49.36% <= 34 milliseconds
52.06% <= 35 milliseconds
54.80% <= 36 milliseconds
57.48% <= 37 milliseconds
60.42% <= 38 milliseconds
63.25% <= 39 milliseconds
65.84% <= 40 milliseconds
68.45% <= 41 milliseconds
71.05% <= 42 milliseconds
73.70% <= 43 milliseconds
76.34% <= 44 milliseconds
78.99% <= 45 milliseconds
81.75% <= 46 milliseconds
84.61% <= 47 milliseconds
87.32% <= 48 milliseconds
89.78% <= 49 milliseconds
91.89% <= 50 milliseconds
93.38% <= 51 milliseconds
94.47% <= 52 milliseconds
95.22% <= 53 milliseconds
95.40% <= 54 milliseconds
95.55% <= 55 milliseconds
95.70% <= 56 milliseconds
95.81% <= 57 milliseconds
95.87% <= 58 milliseconds
95.91% <= 59 milliseconds
95.94% <= 60 milliseconds
95.99% <= 61 milliseconds
96.03% <= 62 milliseconds
96.05% <= 63 milliseconds
96.08% <= 64 milliseconds
96.09% <= 253 milliseconds
96.09% <= 259 milliseconds
96.12% <= 260 milliseconds
96.13% <= 261 milliseconds
96.14% <= 262 milliseconds
96.17% <= 263 milliseconds
96.20% <= 264 milliseconds
96.23% <= 265 milliseconds
96.29% <= 266 milliseconds
96.35% <= 267 milliseconds
96.40% <= 268 milliseconds
96.47% <= 269 milliseconds
96.52% <= 270 milliseconds
96.60% <= 271 milliseconds
96.68% <= 272 milliseconds
96.76% <= 273 milliseconds
96.81% <= 274 milliseconds
96.87% <= 275 milliseconds
96.92% <= 276 milliseconds
96.96% <= 277 milliseconds
97.02% <= 278 milliseconds
97.08% <= 279 milliseconds
97.12% <= 280 milliseconds
97.16% <= 281 milliseconds
97.21% <= 282 milliseconds
97.27% <= 283 milliseconds
97.32% <= 284 milliseconds
97.38% <= 285 milliseconds
97.43% <= 286 milliseconds
97.49% <= 287 milliseconds
97.53% <= 288 milliseconds
97.58% <= 289 milliseconds
97.64% <= 290 milliseconds
97.68% <= 291 milliseconds
97.72% <= 292 milliseconds
97.75% <= 293 milliseconds
97.78% <= 294 milliseconds
97.80% <= 295 milliseconds
97.82% <= 296 milliseconds
97.84% <= 297 milliseconds
97.85% <= 298 milliseconds
97.87% <= 299 milliseconds
97.88% <= 300 milliseconds
97.89% <= 301 milliseconds
97.92% <= 302 milliseconds
97.93% <= 303 milliseconds
97.94% <= 304 milliseconds
97.95% <= 305 milliseconds
97.96% <= 306 milliseconds
97.96% <= 307 milliseconds
97.97% <= 308 milliseconds
97.98% <= 309 milliseconds
97.98% <= 310 milliseconds
97.99% <= 311 milliseconds
98.00% <= 312 milliseconds
98.00% <= 313 milliseconds
98.00% <= 315 milliseconds
98.01% <= 316 milliseconds
98.01% <= 317 milliseconds
98.02% <= 318 milliseconds
98.02% <= 320 milliseconds
98.02% <= 321 milliseconds
98.03% <= 322 milliseconds
98.03% <= 323 milliseconds
98.03% <= 326 milliseconds
98.04% <= 327 milliseconds
98.04% <= 328 milliseconds
98.04% <= 830 milliseconds
98.04% <= 832 milliseconds
98.05% <= 834 milliseconds
98.05% <= 835 milliseconds
98.06% <= 836 milliseconds
98.06% <= 837 milliseconds
98.08% <= 838 milliseconds
98.10% <= 839 milliseconds
98.10% <= 840 milliseconds
98.11% <= 841 milliseconds
98.12% <= 842 milliseconds
98.13% <= 843 milliseconds
98.14% <= 844 milliseconds
98.15% <= 845 milliseconds
98.16% <= 846 milliseconds
98.17% <= 847 milliseconds
98.19% <= 848 milliseconds
98.21% <= 849 milliseconds
98.24% <= 850 milliseconds
98.26% <= 851 milliseconds
98.28% <= 852 milliseconds
98.31% <= 853 milliseconds
98.33% <= 854 milliseconds
98.35% <= 855 milliseconds
98.39% <= 856 milliseconds
98.44% <= 857 milliseconds
98.47% <= 858 milliseconds
98.50% <= 859 milliseconds
98.53% <= 860 milliseconds
98.55% <= 861 milliseconds
98.58% <= 862 milliseconds
98.61% <= 863 milliseconds
98.64% <= 864 milliseconds
98.68% <= 865 milliseconds
98.71% <= 866 milliseconds
98.74% <= 867 milliseconds
98.77% <= 868 milliseconds
98.80% <= 869 milliseconds
98.85% <= 870 milliseconds
98.89% <= 871 milliseconds
98.92% <= 872 milliseconds
98.96% <= 873 milliseconds
98.97% <= 874 milliseconds
99.00% <= 875 milliseconds
99.03% <= 876 milliseconds
99.05% <= 877 milliseconds
99.07% <= 878 milliseconds
99.08% <= 879 milliseconds
99.09% <= 880 milliseconds
99.12% <= 881 milliseconds
99.15% <= 882 milliseconds
99.18% <= 883 milliseconds
99.22% <= 884 milliseconds
99.26% <= 885 milliseconds
99.27% <= 886 milliseconds
99.29% <= 887 milliseconds
99.30% <= 888 milliseconds
99.32% <= 889 milliseconds
99.33% <= 890 milliseconds
99.35% <= 891 milliseconds
99.36% <= 892 milliseconds
99.38% <= 893 milliseconds
99.40% <= 894 milliseconds
99.41% <= 895 milliseconds
99.42% <= 896 milliseconds
99.43% <= 897 milliseconds
99.45% <= 898 milliseconds
99.48% <= 899 milliseconds
99.51% <= 900 milliseconds
99.54% <= 901 milliseconds
99.58% <= 902 milliseconds
99.61% <= 903 milliseconds
99.64% <= 904 milliseconds
99.67% <= 905 milliseconds
99.71% <= 906 milliseconds
99.75% <= 907 milliseconds
99.78% <= 908 milliseconds
99.82% <= 909 milliseconds
99.85% <= 910 milliseconds
99.86% <= 911 milliseconds
99.86% <= 913 milliseconds
99.87% <= 915 milliseconds
99.87% <= 916 milliseconds
99.88% <= 918 milliseconds
99.88% <= 919 milliseconds
99.89% <= 920 milliseconds
99.90% <= 921 milliseconds
99.91% <= 922 milliseconds
99.91% <= 924 milliseconds
99.92% <= 927 milliseconds
99.92% <= 928 milliseconds
99.92% <= 930 milliseconds
99.93% <= 931 milliseconds
99.93% <= 932 milliseconds
99.93% <= 934 milliseconds
99.93% <= 935 milliseconds
99.94% <= 936 milliseconds
99.95% <= 937 milliseconds
99.95% <= 938 milliseconds
99.97% <= 939 milliseconds
99.97% <= 940 milliseconds
99.98% <= 941 milliseconds
99.98% <= 942 milliseconds
99.99% <= 943 milliseconds
100.00% <= 943 milliseconds
2236136.00 requests per second

[tidwall]

Thanks for reporting this. Interesting. I'm going to dig a little.

[tidwall]

I'm seeing different results than you are. In my testing both Redis and Redcon are pretty close as far as latency is concerned.

I used go run example/clone.go for Redcon.

My go version is go1.7 darwin/amd64.

Redis

redis-benchmark -h 127.0.0.1 -p 6379 -t set -n 10000000 -P 1 -c 768
====== SET ======
  10000000 requests completed in 91.12 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 1 milliseconds
0.00% <= 2 milliseconds
0.06% <= 3 milliseconds
18.77% <= 4 milliseconds
26.38% <= 5 milliseconds
35.45% <= 6 milliseconds
89.16% <= 7 milliseconds
97.15% <= 8 milliseconds
98.57% <= 9 milliseconds
99.41% <= 10 milliseconds
99.87% <= 11 milliseconds
99.93% <= 12 milliseconds
99.95% <= 13 milliseconds
99.96% <= 14 milliseconds
99.97% <= 15 milliseconds
99.97% <= 16 milliseconds
99.98% <= 17 milliseconds
99.98% <= 18 milliseconds
99.98% <= 19 milliseconds
99.98% <= 20 milliseconds
99.98% <= 21 milliseconds
99.99% <= 22 milliseconds
99.99% <= 23 milliseconds
99.99% <= 24 milliseconds
99.99% <= 25 milliseconds
99.99% <= 26 milliseconds
99.99% <= 27 milliseconds
99.99% <= 28 milliseconds
99.99% <= 29 milliseconds
99.99% <= 30 milliseconds
99.99% <= 31 milliseconds
99.99% <= 32 milliseconds
99.99% <= 33 milliseconds
99.99% <= 34 milliseconds
99.99% <= 35 milliseconds
99.99% <= 36 milliseconds
99.99% <= 37 milliseconds
99.99% <= 38 milliseconds
99.99% <= 39 milliseconds
99.99% <= 40 milliseconds
99.99% <= 41 milliseconds
99.99% <= 42 milliseconds
99.99% <= 43 milliseconds
99.99% <= 44 milliseconds
99.99% <= 45 milliseconds
99.99% <= 46 milliseconds
100.00% <= 47 milliseconds
100.00% <= 48 milliseconds
100.00% <= 49 milliseconds
100.00% <= 49 milliseconds
109739.37 requests per second

redis-benchmark -h 127.0.0.1 -p 6379 -t set -n 10000000 -P 256 -c 768
====== SET ======
  10000000 requests completed in 10.47 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 5 milliseconds
0.01% <= 15 milliseconds
0.01% <= 16 milliseconds
0.01% <= 17 milliseconds
0.02% <= 18 milliseconds
0.02% <= 19 milliseconds
0.03% <= 20 milliseconds
0.03% <= 21 milliseconds
0.04% <= 22 milliseconds
0.09% <= 23 milliseconds
0.10% <= 24 milliseconds
0.11% <= 25 milliseconds
0.12% <= 26 milliseconds
0.13% <= 27 milliseconds
0.14% <= 28 milliseconds
0.19% <= 29 milliseconds
0.25% <= 30 milliseconds
0.30% <= 31 milliseconds
0.36% <= 32 milliseconds
0.41% <= 33 milliseconds
0.46% <= 34 milliseconds
0.46% <= 35 milliseconds
0.47% <= 36 milliseconds
0.49% <= 37 milliseconds
0.50% <= 38 milliseconds
0.52% <= 39 milliseconds
0.52% <= 40 milliseconds
0.53% <= 41 milliseconds
0.54% <= 42 milliseconds
0.54% <= 43 milliseconds
0.55% <= 44 milliseconds
0.55% <= 45 milliseconds
0.56% <= 46 milliseconds
0.56% <= 47 milliseconds
0.56% <= 48 milliseconds
0.57% <= 49 milliseconds
0.57% <= 50 milliseconds
0.58% <= 51 milliseconds
0.58% <= 52 milliseconds
0.59% <= 53 milliseconds
0.59% <= 73 milliseconds
0.60% <= 75 milliseconds
0.60% <= 80 milliseconds
0.60% <= 81 milliseconds
0.61% <= 82 milliseconds
0.61% <= 85 milliseconds
0.62% <= 86 milliseconds
0.62% <= 87 milliseconds
0.63% <= 88 milliseconds
0.64% <= 89 milliseconds
0.66% <= 90 milliseconds
0.67% <= 91 milliseconds
0.68% <= 92 milliseconds
0.70% <= 93 milliseconds
0.73% <= 94 milliseconds
0.76% <= 95 milliseconds
0.78% <= 96 milliseconds
0.82% <= 97 milliseconds
0.86% <= 98 milliseconds
0.89% <= 99 milliseconds
0.94% <= 100 milliseconds
0.95% <= 101 milliseconds
0.96% <= 102 milliseconds
0.97% <= 103 milliseconds
0.98% <= 104 milliseconds
0.99% <= 105 milliseconds
0.99% <= 108 milliseconds
0.99% <= 113 milliseconds
0.99% <= 114 milliseconds
1.00% <= 115 milliseconds
1.00% <= 117 milliseconds
1.00% <= 118 milliseconds
1.01% <= 119 milliseconds
1.01% <= 120 milliseconds
1.01% <= 121 milliseconds
1.02% <= 122 milliseconds
1.03% <= 123 milliseconds
1.09% <= 124 milliseconds
1.14% <= 125 milliseconds
1.19% <= 126 milliseconds
1.24% <= 127 milliseconds
1.30% <= 128 milliseconds
1.35% <= 129 milliseconds
1.41% <= 130 milliseconds
1.46% <= 131 milliseconds
1.52% <= 132 milliseconds
1.57% <= 133 milliseconds
1.63% <= 134 milliseconds
1.68% <= 135 milliseconds
1.73% <= 136 milliseconds
1.79% <= 137 milliseconds
1.85% <= 138 milliseconds
1.91% <= 139 milliseconds
1.97% <= 140 milliseconds
2.04% <= 141 milliseconds
2.10% <= 142 milliseconds
2.18% <= 143 milliseconds
2.23% <= 144 milliseconds
2.29% <= 145 milliseconds
2.36% <= 146 milliseconds
2.41% <= 147 milliseconds
2.43% <= 148 milliseconds
2.46% <= 149 milliseconds
2.47% <= 150 milliseconds
2.47% <= 151 milliseconds
2.48% <= 152 milliseconds
2.49% <= 153 milliseconds
2.50% <= 154 milliseconds
2.50% <= 172 milliseconds
2.54% <= 173 milliseconds
2.59% <= 174 milliseconds
2.74% <= 175 milliseconds
2.99% <= 176 milliseconds
3.28% <= 177 milliseconds
3.68% <= 178 milliseconds
4.39% <= 179 milliseconds
5.37% <= 180 milliseconds
6.69% <= 181 milliseconds
8.23% <= 182 milliseconds
9.96% <= 183 milliseconds
11.96% <= 184 milliseconds
14.21% <= 185 milliseconds
16.74% <= 186 milliseconds
19.37% <= 187 milliseconds
22.19% <= 188 milliseconds
25.27% <= 189 milliseconds
28.63% <= 190 milliseconds
32.37% <= 191 milliseconds
36.21% <= 192 milliseconds
40.23% <= 193 milliseconds
44.50% <= 194 milliseconds
48.81% <= 195 milliseconds
52.96% <= 196 milliseconds
57.15% <= 197 milliseconds
61.15% <= 198 milliseconds
65.06% <= 199 milliseconds
68.79% <= 200 milliseconds
72.35% <= 201 milliseconds
75.67% <= 202 milliseconds
78.65% <= 203 milliseconds
81.13% <= 204 milliseconds
83.28% <= 205 milliseconds
85.18% <= 206 milliseconds
86.81% <= 207 milliseconds
88.10% <= 208 milliseconds
89.21% <= 209 milliseconds
90.27% <= 210 milliseconds
91.25% <= 211 milliseconds
92.00% <= 212 milliseconds
92.69% <= 213 milliseconds
93.36% <= 214 milliseconds
93.97% <= 215 milliseconds
94.57% <= 216 milliseconds
95.13% <= 217 milliseconds
95.65% <= 218 milliseconds
96.13% <= 219 milliseconds
96.56% <= 220 milliseconds
96.95% <= 221 milliseconds
97.27% <= 222 milliseconds
97.55% <= 223 milliseconds
97.81% <= 224 milliseconds
98.03% <= 225 milliseconds
98.21% <= 226 milliseconds
98.38% <= 227 milliseconds
98.54% <= 228 milliseconds
98.70% <= 229 milliseconds
98.82% <= 230 milliseconds
98.95% <= 231 milliseconds
99.06% <= 232 milliseconds
99.16% <= 233 milliseconds
99.26% <= 234 milliseconds
99.36% <= 235 milliseconds
99.46% <= 236 milliseconds
99.56% <= 237 milliseconds
99.65% <= 238 milliseconds
99.74% <= 239 milliseconds
99.78% <= 240 milliseconds
99.83% <= 241 milliseconds
99.88% <= 242 milliseconds
99.93% <= 243 milliseconds
99.98% <= 244 milliseconds
100.00% <= 244 milliseconds
955018.62 requests per second

Redcon

redis-benchmark -h 127.0.0.1 -p 6380 -t set -n 10000000 -P 1 -c 768
====== SET ======
  10000000 requests completed in 109.67 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
0.00% <= 3 milliseconds
1.52% <= 4 milliseconds
5.26% <= 5 milliseconds
9.26% <= 6 milliseconds
21.51% <= 7 milliseconds
66.51% <= 8 milliseconds
83.52% <= 9 milliseconds
89.89% <= 10 milliseconds
96.50% <= 11 milliseconds
99.21% <= 12 milliseconds
99.79% <= 13 milliseconds
99.94% <= 14 milliseconds
99.99% <= 15 milliseconds
100.00% <= 16 milliseconds
100.00% <= 17 milliseconds
100.00% <= 18 milliseconds
100.00% <= 19 milliseconds
100.00% <= 20 milliseconds
100.00% <= 21 milliseconds
100.00% <= 21 milliseconds
91180.98 requests per second

redis-benchmark -h 127.0.0.1 -p 6380 -t set -n 10000000 -P 256 -c 768
====== SET ======
  10000000 requests completed in 4.56 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.26% <= 1 milliseconds
0.61% <= 2 milliseconds
0.93% <= 3 milliseconds
1.22% <= 4 milliseconds
1.49% <= 5 milliseconds
1.77% <= 6 milliseconds
2.05% <= 7 milliseconds
2.40% <= 8 milliseconds
2.74% <= 9 milliseconds
3.07% <= 10 milliseconds
3.40% <= 11 milliseconds
3.70% <= 12 milliseconds
3.98% <= 13 milliseconds
4.24% <= 14 milliseconds
4.54% <= 15 milliseconds
4.85% <= 16 milliseconds
5.11% <= 17 milliseconds
5.36% <= 18 milliseconds
5.58% <= 19 milliseconds
5.83% <= 20 milliseconds
6.09% <= 21 milliseconds
6.36% <= 22 milliseconds
6.62% <= 23 milliseconds
6.86% <= 24 milliseconds
7.13% <= 25 milliseconds
7.43% <= 26 milliseconds
7.69% <= 27 milliseconds
8.00% <= 28 milliseconds
8.32% <= 29 milliseconds
9.02% <= 30 milliseconds
9.42% <= 31 milliseconds
9.78% <= 32 milliseconds
10.17% <= 33 milliseconds
10.55% <= 34 milliseconds
10.88% <= 35 milliseconds
11.24% <= 36 milliseconds
11.63% <= 37 milliseconds
12.03% <= 38 milliseconds
12.45% <= 39 milliseconds
12.83% <= 40 milliseconds
13.27% <= 41 milliseconds
13.73% <= 42 milliseconds
14.11% <= 43 milliseconds
14.58% <= 44 milliseconds
14.98% <= 45 milliseconds
15.44% <= 46 milliseconds
15.90% <= 47 milliseconds
16.43% <= 48 milliseconds
16.96% <= 49 milliseconds
17.48% <= 50 milliseconds
18.01% <= 51 milliseconds
18.61% <= 52 milliseconds
19.16% <= 53 milliseconds
19.83% <= 54 milliseconds
20.45% <= 55 milliseconds
21.09% <= 56 milliseconds
21.68% <= 57 milliseconds
22.36% <= 58 milliseconds
23.08% <= 59 milliseconds
23.73% <= 60 milliseconds
24.47% <= 61 milliseconds
25.25% <= 62 milliseconds
26.05% <= 63 milliseconds
26.90% <= 64 milliseconds
27.71% <= 65 milliseconds
28.52% <= 66 milliseconds
29.43% <= 67 milliseconds
30.42% <= 68 milliseconds
31.45% <= 69 milliseconds
32.46% <= 70 milliseconds
33.48% <= 71 milliseconds
34.57% <= 72 milliseconds
35.71% <= 73 milliseconds
36.85% <= 74 milliseconds
38.11% <= 75 milliseconds
39.27% <= 76 milliseconds
40.50% <= 77 milliseconds
41.68% <= 78 milliseconds
43.00% <= 79 milliseconds
44.35% <= 80 milliseconds
45.70% <= 81 milliseconds
47.10% <= 82 milliseconds
48.51% <= 83 milliseconds
49.77% <= 84 milliseconds
51.14% <= 85 milliseconds
52.56% <= 86 milliseconds
54.00% <= 87 milliseconds
55.59% <= 88 milliseconds
57.03% <= 89 milliseconds
58.52% <= 90 milliseconds
59.93% <= 91 milliseconds
61.37% <= 92 milliseconds
62.77% <= 93 milliseconds
64.25% <= 94 milliseconds
65.86% <= 95 milliseconds
67.33% <= 96 milliseconds
68.85% <= 97 milliseconds
70.35% <= 98 milliseconds
71.76% <= 99 milliseconds
73.06% <= 100 milliseconds
74.40% <= 101 milliseconds
75.85% <= 102 milliseconds
77.23% <= 103 milliseconds
78.49% <= 104 milliseconds
79.68% <= 105 milliseconds
80.91% <= 106 milliseconds
82.08% <= 107 milliseconds
83.22% <= 108 milliseconds
84.38% <= 109 milliseconds
85.43% <= 110 milliseconds
86.46% <= 111 milliseconds
87.41% <= 112 milliseconds
88.30% <= 113 milliseconds
89.21% <= 114 milliseconds
89.97% <= 115 milliseconds
90.81% <= 116 milliseconds
91.59% <= 117 milliseconds
92.27% <= 118 milliseconds
92.88% <= 119 milliseconds
93.48% <= 120 milliseconds
94.04% <= 121 milliseconds
94.58% <= 122 milliseconds
95.09% <= 123 milliseconds
95.51% <= 124 milliseconds
95.91% <= 125 milliseconds
96.31% <= 126 milliseconds
96.62% <= 127 milliseconds
96.95% <= 128 milliseconds
97.23% <= 129 milliseconds
97.49% <= 130 milliseconds
97.71% <= 131 milliseconds
97.99% <= 132 milliseconds
98.21% <= 133 milliseconds
98.41% <= 134 milliseconds
98.56% <= 135 milliseconds
98.70% <= 136 milliseconds
98.86% <= 137 milliseconds
98.95% <= 138 milliseconds
99.06% <= 139 milliseconds
99.17% <= 140 milliseconds
99.27% <= 141 milliseconds
99.34% <= 142 milliseconds
99.43% <= 143 milliseconds
99.49% <= 144 milliseconds
99.55% <= 145 milliseconds
99.58% <= 146 milliseconds
99.62% <= 147 milliseconds
99.67% <= 148 milliseconds
99.69% <= 149 milliseconds
99.72% <= 150 milliseconds
99.75% <= 151 milliseconds
99.77% <= 152 milliseconds
99.81% <= 153 milliseconds
99.82% <= 154 milliseconds
99.84% <= 155 milliseconds
99.85% <= 156 milliseconds
99.85% <= 157 milliseconds
99.86% <= 158 milliseconds
99.87% <= 159 milliseconds
99.88% <= 160 milliseconds
99.91% <= 161 milliseconds
99.92% <= 162 milliseconds
99.93% <= 163 milliseconds
99.94% <= 164 milliseconds
99.95% <= 165 milliseconds
99.95% <= 166 milliseconds
99.96% <= 167 milliseconds
99.96% <= 168 milliseconds
99.97% <= 170 milliseconds
99.98% <= 171 milliseconds
99.98% <= 176 milliseconds
99.98% <= 177 milliseconds
99.99% <= 178 milliseconds
99.99% <= 182 milliseconds
99.99% <= 183 milliseconds
99.99% <= 189 milliseconds
100.00% <= 190 milliseconds
100.00% <= 190 milliseconds
2193463.50 requests per second

[simongui]

I'm benchmarking between two 20 core servers on 10GbE. That's a localhost test. Do you have 2 machines you can benchmark with over a real network?

[tidwall]

I have a couple machines I can test with. I'll keep you posted. Is there anything specific that I should configure for?

[simongui]

I'm just testing on 2 20 core dual 10GbE bonded to 20GbE servers. I'll keep testing as well to make sure it's not an anomaly.

[simongui]

In this benchmark the latency is close at the 99.6% range but then closer to MAX it starts to really spike.

Redis

redis-benchmark -h server -p 6379 -t set -n 10000000 -P 128 -c 128
====== SET ======
  10000000 requests completed in 8.06 seconds
  128 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 1 milliseconds
0.02% <= 2 milliseconds
0.02% <= 3 milliseconds
0.02% <= 4 milliseconds
0.03% <= 5 milliseconds
0.03% <= 6 milliseconds
0.05% <= 7 milliseconds
0.21% <= 8 milliseconds
0.31% <= 9 milliseconds
2.86% <= 10 milliseconds
22.38% <= 11 milliseconds
44.57% <= 12 milliseconds
76.49% <= 13 milliseconds
92.29% <= 14 milliseconds
97.15% <= 15 milliseconds
99.83% <= 16 milliseconds
99.84% <= 31 milliseconds
99.86% <= 32 milliseconds
99.88% <= 33 milliseconds
99.91% <= 34 milliseconds
99.93% <= 35 milliseconds
99.96% <= 36 milliseconds
99.99% <= 37 milliseconds
100.00% <= 37 milliseconds
1240694.75 requests per second

Redcon

redis-benchmark -h server -p 6380 -t set -n 10000000 -P 128 -c 128
====== SET ======
  10000000 requests completed in 4.12 seconds
  128 parallel clients
  3 bytes payload
  keep alive: 1

0.04% <= 1 milliseconds
64.78% <= 2 milliseconds
78.01% <= 3 milliseconds
97.67% <= 4 milliseconds
99.45% <= 5 milliseconds
99.59% <= 6 milliseconds
99.65% <= 7 milliseconds
99.66% <= 8 milliseconds
99.66% <= 9 milliseconds
99.66% <= 11 milliseconds
99.67% <= 13 milliseconds
99.67% <= 19 milliseconds
99.67% <= 867 milliseconds
99.67% <= 868 milliseconds
99.68% <= 869 milliseconds
99.70% <= 870 milliseconds
99.73% <= 871 milliseconds
99.75% <= 872 milliseconds
99.77% <= 873 milliseconds
99.81% <= 874 milliseconds
99.83% <= 875 milliseconds
99.87% <= 876 milliseconds
99.90% <= 877 milliseconds
99.93% <= 878 milliseconds
99.95% <= 879 milliseconds
99.95% <= 880 milliseconds
99.96% <= 881 milliseconds
99.96% <= 882 milliseconds
99.96% <= 883 milliseconds
99.97% <= 884 milliseconds
99.98% <= 885 milliseconds
99.98% <= 886 milliseconds
99.99% <= 887 milliseconds
99.99% <= 888 milliseconds
99.99% <= 889 milliseconds
99.99% <= 890 milliseconds
100.00% <= 891 milliseconds
100.00% <= 900 milliseconds
100.00% <= 900 milliseconds
2424830.25 requests per second

[tidwall]

At home I only have a one linux box and my macbook, and I couldn't find my thunderbolt/ethernet adapter. 👎 So fuck it. I ended up spinning up two servers on Digital Ocean. Both 2GB instances with private networking. I compiled Redis on both servers so that they both have the latest redis-server and redis-benchmark commands.

On Server1 I launched Redis with src/redis-server --protected-mode no and Redcon with go run examples\clone.go.

On Server 2 I ran the redis-benchmark utility.

Redis

redis-benchmark -h 10.134.22.215 -p 6379 -t set -n 10000000 -P 128 -c 128
====== SET ======
  10000000 requests completed in 16.08 seconds
  128 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
0.00% <= 3 milliseconds
0.01% <= 6 milliseconds
0.03% <= 7 milliseconds
0.04% <= 11 milliseconds
0.05% <= 12 milliseconds
0.08% <= 13 milliseconds
0.12% <= 14 milliseconds
0.13% <= 15 milliseconds
0.14% <= 16 milliseconds
0.19% <= 17 milliseconds
0.31% <= 18 milliseconds
0.60% <= 19 milliseconds
1.56% <= 20 milliseconds
4.38% <= 21 milliseconds
11.69% <= 22 milliseconds
26.43% <= 23 milliseconds
47.82% <= 24 milliseconds
67.76% <= 25 milliseconds
79.92% <= 26 milliseconds
86.80% <= 27 milliseconds
90.36% <= 28 milliseconds
92.59% <= 29 milliseconds
94.24% <= 30 milliseconds
95.64% <= 31 milliseconds
96.75% <= 32 milliseconds
97.61% <= 33 milliseconds
98.08% <= 34 milliseconds
98.40% <= 35 milliseconds
98.67% <= 36 milliseconds
98.88% <= 37 milliseconds
99.06% <= 38 milliseconds
99.20% <= 39 milliseconds
99.30% <= 40 milliseconds
99.38% <= 41 milliseconds
99.45% <= 42 milliseconds
99.50% <= 43 milliseconds
99.55% <= 44 milliseconds
99.61% <= 45 milliseconds
99.69% <= 46 milliseconds
99.77% <= 47 milliseconds
99.83% <= 48 milliseconds
99.87% <= 49 milliseconds
99.90% <= 50 milliseconds
99.92% <= 51 milliseconds
99.92% <= 52 milliseconds
99.93% <= 53 milliseconds
99.96% <= 54 milliseconds
99.98% <= 55 milliseconds
99.99% <= 56 milliseconds
100.00% <= 56 milliseconds
622045.25 requests per second

redis-benchmark -h 10.134.22.215 -p 6380 -t set -n 10000000 -P 128 -c 128
====== SET ======
  10000000 requests completed in 8.20 seconds
  128 parallel clients
  3 bytes payload
  keep alive: 1

0.47% <= 1 milliseconds
1.89% <= 2 milliseconds
3.99% <= 3 milliseconds
6.42% <= 4 milliseconds
9.19% <= 5 milliseconds
12.33% <= 6 milliseconds
16.56% <= 7 milliseconds
21.64% <= 8 milliseconds
27.29% <= 9 milliseconds
33.50% <= 10 milliseconds
40.06% <= 11 milliseconds
46.80% <= 12 milliseconds
53.39% <= 13 milliseconds
59.58% <= 14 milliseconds
65.37% <= 15 milliseconds
70.32% <= 16 milliseconds
74.92% <= 17 milliseconds
79.12% <= 18 milliseconds
82.85% <= 19 milliseconds
86.44% <= 20 milliseconds
89.49% <= 21 milliseconds
91.76% <= 22 milliseconds
93.33% <= 23 milliseconds
94.43% <= 24 milliseconds
95.22% <= 25 milliseconds
95.91% <= 26 milliseconds
96.48% <= 27 milliseconds
96.98% <= 28 milliseconds
97.45% <= 29 milliseconds
97.83% <= 30 milliseconds
98.16% <= 31 milliseconds
98.42% <= 32 milliseconds
98.65% <= 33 milliseconds
98.84% <= 34 milliseconds
98.98% <= 35 milliseconds
99.09% <= 36 milliseconds
99.18% <= 37 milliseconds
99.28% <= 38 milliseconds
99.36% <= 39 milliseconds
99.42% <= 40 milliseconds
99.48% <= 41 milliseconds
99.53% <= 42 milliseconds
99.59% <= 43 milliseconds
99.64% <= 44 milliseconds
99.67% <= 45 milliseconds
99.72% <= 46 milliseconds
99.74% <= 47 milliseconds
99.77% <= 48 milliseconds
99.78% <= 49 milliseconds
99.80% <= 50 milliseconds
99.80% <= 51 milliseconds
99.82% <= 52 milliseconds
99.83% <= 53 milliseconds
99.84% <= 54 milliseconds
99.85% <= 55 milliseconds
99.86% <= 56 milliseconds
99.87% <= 57 milliseconds
99.88% <= 58 milliseconds
99.89% <= 59 milliseconds
99.89% <= 60 milliseconds
99.90% <= 61 milliseconds
99.90% <= 63 milliseconds
99.91% <= 64 milliseconds
99.91% <= 65 milliseconds
99.91% <= 66 milliseconds
99.92% <= 67 milliseconds
99.92% <= 68 milliseconds
99.92% <= 69 milliseconds
99.93% <= 70 milliseconds
99.94% <= 71 milliseconds
99.94% <= 72 milliseconds
99.94% <= 73 milliseconds
99.95% <= 74 milliseconds
99.95% <= 75 milliseconds
99.95% <= 76 milliseconds
99.96% <= 77 milliseconds
99.96% <= 79 milliseconds
99.96% <= 80 milliseconds
99.96% <= 81 milliseconds
99.97% <= 82 milliseconds
99.97% <= 83 milliseconds
99.97% <= 84 milliseconds
99.97% <= 86 milliseconds
99.98% <= 87 milliseconds
99.98% <= 88 milliseconds
99.98% <= 89 milliseconds
99.98% <= 91 milliseconds
99.98% <= 92 milliseconds
99.98% <= 93 milliseconds
99.99% <= 95 milliseconds
99.99% <= 97 milliseconds
99.99% <= 98 milliseconds
99.99% <= 100 milliseconds
99.99% <= 104 milliseconds
99.99% <= 107 milliseconds
100.00% <= 109 milliseconds
100.00% <= 118 milliseconds
100.00% <= 149 milliseconds
100.00% <= 149 milliseconds
1219363.50 requests per second

[tidwall]

Now when I run the Redcon clone as a single-theaded, the results are more in-line with Redis.

GOMAXPROCS=1 go run example/clone.go

Redcon

redis-benchmark -h 10.134.22.215 -p 6380 -t set -n 10000000 -P 128 -c 128
====== SET ======
  10000000 requests completed in 14.53 seconds
  128 parallel clients
  3 bytes payload
  keep alive: 1

0.02% <= 1 milliseconds
0.10% <= 2 milliseconds
0.17% <= 3 milliseconds
0.25% <= 4 milliseconds
0.33% <= 5 milliseconds
0.41% <= 6 milliseconds
0.52% <= 7 milliseconds
0.61% <= 8 milliseconds
0.72% <= 9 milliseconds
0.82% <= 10 milliseconds
1.04% <= 11 milliseconds
1.80% <= 12 milliseconds
3.13% <= 13 milliseconds
5.38% <= 14 milliseconds
8.55% <= 15 milliseconds
12.33% <= 16 milliseconds
16.46% <= 17 milliseconds
20.89% <= 18 milliseconds
25.57% <= 19 milliseconds
30.40% <= 20 milliseconds
35.39% <= 21 milliseconds
40.54% <= 22 milliseconds
45.89% <= 23 milliseconds
51.22% <= 24 milliseconds
56.69% <= 25 milliseconds
62.00% <= 26 milliseconds
67.21% <= 27 milliseconds
72.26% <= 28 milliseconds
77.12% <= 29 milliseconds
81.87% <= 30 milliseconds
86.44% <= 31 milliseconds
90.42% <= 32 milliseconds
93.72% <= 33 milliseconds
96.10% <= 34 milliseconds
97.55% <= 35 milliseconds
98.41% <= 36 milliseconds
98.91% <= 37 milliseconds
99.21% <= 38 milliseconds
99.42% <= 39 milliseconds
99.57% <= 40 milliseconds
99.68% <= 41 milliseconds
99.77% <= 42 milliseconds
99.84% <= 43 milliseconds
99.90% <= 44 milliseconds
99.93% <= 45 milliseconds
99.97% <= 46 milliseconds
99.98% <= 47 milliseconds
99.99% <= 48 milliseconds
100.00% <= 50 milliseconds
100.00% <= 54 milliseconds
100.00% <= 55 milliseconds
100.00% <= 55 milliseconds
687994.50 requests per second

[simongui]

Interesting results! I wonder where the contention is that is causing the tail latency to spike so much.

[simongui]

Yeah I think something else in my codebase may be interfering. I'm benchmarking with your clone code and it's working more closely to Redis. Sorry for wasting some of your time. I'll dig into what's going on. I don't see anything consuming much CPU usage but there must be something interfering.

[tidwall]

Yeah. I'm curious too.

Maybe there's some network setting in /etc/sysctl.conf (or elsewhere) that is conflicting with the something in the Go net package?

Another thought is around the 20-core processor server. Perhaps try with GOMAXPROCS=1?

[tidwall]

Our messages crossed paths. I'm glad to hear that it may something that's fixable. It's totally not a waste of time. I'm more than happy to help investigate these types of things.

Thanks for using the project and let me know if there's anything I can help with.

[simongui]

I found the problem. Add this to clone.go

type keyvaluepair struct {
    bucket []byte
    key    []byte
    value  []byte
}
var writes chan keyvaluepair

Then at the top of main() add the following. Compile and benchmark.

writes = make(chan keyvaluepair, 10000000)

I'm not sure why this is causing issues because the channel isn't being used.

[tidwall]

It's happening on my side too. It's definitely something to do with the huge channel. I changed the size from 10,000,000 to 100,000 and it sped it up quite a bit.

That channel sucks up 2GB of ram. 10,000,000 elements * 216 byte struct.

[simongui]

Yeah. I was going to synchronize writes using a channel, but since sometimes Redcon can reach 2 million+ requests/second I thought I would give it a channel of 10 million to give it some room to not block.

[tidwall]

Perhaps a sync.RWMutex would be better. You could possibly have mu.Lock() that wraps Redcon commands that write and mu.RLock() wrap commands that read. That's the pattern that I use and it seems pretty good.