Closed SiarheiFedartsou closed 5 months ago
It was low hanging fruit as we are running these tools for other benchmarks anyway.
Also I noticed that e2e tests I added recently are very "jumpy" :( I will probably address it in some separate PR...
It was low hanging fruit as we are running these tools for other benchmarks anyway.
Also I noticed that e2e tests I added recently are very "jumpy" :( I will probably address it in some separate PR...
| Benchmark | Base | PR | |-----------|------|----| | alias | aliased u32: 1133.92Benchmark Results
plain u32: 1133.8
aliased double: 1167.26
plain double: 1169.79 | aliased u32: 1097.71
plain u32: 1138.42
aliased double: 1172.87
plain double: 1179.5 | | e2e_match_ch | requests: 399
failures: 0
req/s: 6.650req/s
avg: 5.575ms
50%: 3ms
75%: 3ms
95%: 25ms
98%: 49ms
99%: 62ms
min: 1.157ms
max: 96.289ms | requests: 385
failures: 0
req/s: 6.425req/s
avg: 5.557ms
50%: 3ms
75%: 4ms
95%: 19ms
98%: 46ms
99%: 70ms
min: 1.464ms
max: 91.964ms | | e2e_match_mld | requests: 362
failures: 0
req/s: 6.034req/s
avg: 4.496ms
50%: 3ms
75%: 4ms
95%: 11ms
98%: 19ms
99%: 53ms
min: 1.013ms
max: 66.555ms | requests: 367
failures: 0
req/s: 6.116req/s
avg: 5.576ms
50%: 3ms
75%: 4ms
95%: 18ms
98%: 52ms
99%: 75ms
min: 1.008ms
max: 87.581ms | | e2e_nearest_ch | requests: 428
failures: 0
req/s: 7.134req/s
avg: 1.420ms
50%: 1ms
75%: 2ms
95%: 2ms
98%: 2ms
99%: 2ms
min: 1.005ms
max: 3.117ms | requests: 422
failures: 0
req/s: 7.042req/s
avg: 1.571ms
50%: 2ms
75%: 2ms
95%: 2ms
98%: 2ms
99%: 3ms
min: 1.068ms
max: 2.761ms | | e2e_nearest_mld | requests: 426
failures: 0
req/s: 7.100req/s
avg: 1.458ms
50%: 1ms
75%: 2ms
95%: 2ms
98%: 2ms
99%: 2ms
min: 1.039ms
max: 6.775ms | requests: 425
failures: 0
req/s: 7.083req/s
avg: 1.524ms
50%: 1ms
75%: 2ms
95%: 2ms
98%: 2ms
99%: 2ms
min: 1.033ms
max: 3.204ms | | e2e_route_ch | requests: 414
failures: 0
req/s: 6.900req/s
avg: 3.662ms
50%: 4ms
75%: 4ms
95%: 5ms
98%: 6ms
99%: 8ms
min: 1.472ms
max: 9.935ms | requests: 403
failures: 0
req/s: 6.725req/s
avg: 4.276ms
50%: 4ms
75%: 5ms
95%: 6ms
98%: 7ms
99%: 10ms
min: 1.470ms
max: 99.512ms | | e2e_route_mld | requests: 372
failures: 0
req/s: 6.200req/s
avg: 4.636ms
50%: 4ms
75%: 5ms
95%: 6ms
98%: 7ms
99%: 10ms
min: 1.373ms
max: 141.264ms | requests: 419
failures: 0
req/s: 6.983req/s
avg: 4.248ms
50%: 4ms
75%: 5ms
95%: 6ms
98%: 7ms
99%: 9ms
min: 1.358ms
max: 97.756ms | | e2e_table_ch | requests: 388
failures: 0
req/s: 6.467req/s
avg: 29.454ms
50%: 22ms
75%: 30ms
95%: 110ms
98%: 120ms
99%: 130ms
min: 2.483ms
max: 138.929ms | requests: 400
failures: 0
req/s: 6.675req/s
avg: 35.443ms
50%: 23ms
75%: 34ms
95%: 120ms
98%: 130ms
99%: 140ms
min: 2.451ms
max: 151.331ms | | e2e_table_mld | requests: 385
failures: 0
req/s: 6.417req/s
avg: 73.161ms
50%: 79ms
75%: 110ms
95%: 130ms
98%: 140ms
99%: 140ms
min: 4.356ms
max: 165.649ms | requests: 390
failures: 0
req/s: 6.500req/s
avg: 42.225ms
50%: 27ms
75%: 61ms
95%: 130ms
98%: 130ms
99%: 140ms
min: 2.491ms
max: 328.336ms | | e2e_trip_ch | requests: 369
failures: 0
req/s: 6.150req/s
avg: 13.448ms
50%: 12ms
75%: 17ms
95%: 27ms
98%: 31ms
99%: 32ms
min: 1.994ms
max: 106.068ms | requests: 352
failures: 0
req/s: 5.874req/s
avg: 14.614ms
50%: 13ms
75%: 19ms
95%: 29ms
98%: 32ms
99%: 34ms
min: 1.829ms
max: 36.972ms | | e2e_trip_mld | requests: 352
failures: 0
req/s: 5.867req/s
avg: 20.310ms
50%: 20ms
75%: 26ms
95%: 34ms
98%: 37ms
99%: 39ms
min: 3.991ms
max: 44.437ms | requests: 351
failures: 0
req/s: 5.850req/s
avg: 14.914ms
50%: 13ms
75%: 19ms
95%: 28ms
98%: 33ms
99%: 36ms
min: 2.381ms
max: 40.495ms | | json-render | String: 6.56025ms
Stringstream: 9.27548ms
Vector: 6.8574ms | String: 6.66464ms
Stringstream: 9.44745ms
Vector: 7.01822ms | | match_ch | Default radius:
4.41197ms/req at 82 coordinate
0.0538045ms/coordinate
Radius 5m:
4.39374ms/req at 82 coordinate
0.0535822ms/coordinate
Radius 10m:
15.0527ms/req at 82 coordinate
0.183569ms/coordinate
Radius 15m:
36.7583ms/req at 82 coordinate
0.448272ms/coordinate
Radius 30m:
314.519ms/req at 82 coordinate
3.8356ms/coordinate | Default radius:
4.41337ms/req at 82 coordinate
0.0538215ms/coordinate
Radius 5m:
4.40282ms/req at 82 coordinate
0.053693ms/coordinate
Radius 10m:
15.0342ms/req at 82 coordinate
0.183344ms/coordinate
Radius 15m:
36.9353ms/req at 82 coordinate
0.45043ms/coordinate
Radius 30m:
313.907ms/req at 82 coordinate
3.82813ms/coordinate | | match_mld | Default radius:
2.80752ms/req at 82 coordinate
0.034238ms/coordinate
Radius 5m:
2.74811ms/req at 82 coordinate
0.0335135ms/coordinate
Radius 10m:
10.1372ms/req at 82 coordinate
0.123625ms/coordinate
Radius 15m:
25.9226ms/req at 82 coordinate
0.31613ms/coordinate
Radius 30m:
305.832ms/req at 82 coordinate
3.72966ms/coordinate | Default radius:
2.77088ms/req at 82 coordinate
0.0337912ms/coordinate
Radius 5m:
2.76309ms/req at 82 coordinate
0.0336963ms/coordinate
Radius 10m:
10.1278ms/req at 82 coordinate
0.12351ms/coordinate
Radius 15m:
26.0479ms/req at 82 coordinate
0.317657ms/coordinate
Radius 30m:
302.263ms/req at 82 coordinate
3.68613ms/coordinate | | osrm_contract | Time: 93.18s Peak RAM: 185.41MB | Time: 94.27s Peak RAM: 185.70MB | | osrm_customize | Time: 1.31s Peak RAM: 115.05MB | Time: 1.30s Peak RAM: 114.91MB | | osrm_extract | Time: 12.18s Peak RAM: 415.62MB | Time: 12.31s Peak RAM: 425.00MB | | osrm_partition | Time: 2.18s Peak RAM: 148.48MB | Time: 2.20s Peak RAM: 148.36MB | | packedvector | random write:
std::vector 11117.9 ms
util::packed_vector 73460.3 ms
slowdown: 6.60738
random read:
std::vector 11015.3 ms
util::packed_vector 29535 ms
slowdown: 2.68128 | random write:
std::vector 9948.07 ms
util::packed_vector 73306.6 ms
slowdown: 7.36892
random read:
std::vector 8460.91 ms
util::packed_vector 30177.8 ms
slowdown: 3.56673 | | route_ch | 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
508.293ms
0.508293ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
350.297ms
0.350297ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
624.345ms
0.624345ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
150.667ms
0.150667ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
97.4145ms
0.0974145ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
131.516ms
0.131516ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
149.903ms
0.149903ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
97.0547ms
0.0970547ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
131.542ms
0.131542ms/req | 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
508.853ms
0.508853ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
349.815ms
0.349815ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
624.741ms
0.624741ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
151.161ms
0.151161ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
96.9316ms
0.0969316ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
132.315ms
0.132315ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
152.045ms
0.152045ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
96.9038ms
0.0969038ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
132.335ms
0.132335ms/req | | route_mld | 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
638.445ms
0.638445ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
439.606ms
0.439606ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
817.78ms
0.81778ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
265.954ms
0.265954ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
162.716ms
0.162716ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
283.928ms
0.283928ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
257.784ms
0.257784ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
161.048ms
0.161048ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
281.935ms
0.281935ms/req | 1000 routes, 3 coordinates, no alternatives, overview=full, steps=true
639.416ms
0.639416ms/req
1000 routes, 2 coordinates, no alternatives, overview=full, steps=true
437.273ms
0.437273ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=full, steps=true
810.431ms
0.810431ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false
256.097ms
0.256097ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false
159.142ms
0.159142ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false
280.538ms
0.280538ms/req
1000 routes, 3 coordinates, no alternatives, overview=false, steps=false, radius=750
259.231ms
0.259231ms/req
1000 routes, 2 coordinates, no alternatives, overview=false, steps=false, radius=750
159.369ms
0.159369ms/req
1000 routes, 2 coordinates, 3 alternatives, overview=false, steps=false, radius=750
282.002ms
0.282002ms/req | | rtree | 1 result:
202.806ms -> 0.0202806 ms/query
10 results:
237.224ms -> 0.0237224 ms/query | 1 result:
209.887ms -> 0.0209887 ms/query
10 results:
246.082ms -> 0.0246082 ms/query |