search

DataDog / datadog-process-agent

Datadog Process Agent
https://datadoghq.com
20 stars 9 forks source link

Remove allocations in ByteKey. Pre-allocate size for connections slice in mergeConnections #295

Closed kevinconaway closed 5 years ago

kevinconaway commented 5 years ago

@DataDog/burrito

Saw mergeConnections as a large source of allocations. Pre-allocating the list should help reduce that.

I also removed some low hanging fruit in ByteKey

BenchmarkOldByteKey-2       20000000            81.0 ns/op        24 B/op          3 allocs/op
BenchmarkNewByteKey-2       50000000            24.8 ns/op         0 B/op          0 allocs/op
sunhay commented 5 years ago

We do have some benchmarks in state_test.go, can you run them and provide output?

sunhay commented 5 years ago

I ran it :)

benchmark                                                        old ns/op     new ns/op     delta
BenchmarkStoreClosedConnection/StoreClosedConnection-100-4       25343         16490         -34.93%
BenchmarkStoreClosedConnection/StoreClosedConnection-1000-4      253324        165318        -34.74%
BenchmarkStoreClosedConnection/StoreClosedConnection-10000-4     2626967       1731048       -34.10%
BenchmarkStoreClosedConnection/StoreClosedConnection-30000-4     9050698       6526981       -27.88%
BenchmarkConnectionsGet/ConnectionsGet-100-0-4                   39169         22956         -41.39%
BenchmarkConnectionsGet/ConnectionsGet-100-50-4                  37792         23454         -37.94%
BenchmarkConnectionsGet/ConnectionsGet-100-100-4                 38884         22894         -41.12%
BenchmarkConnectionsGet/ConnectionsGet-1000-0-4                  365657        248599        -32.01%
BenchmarkConnectionsGet/ConnectionsGet-1000-500-4                354532        235007        -33.71%
BenchmarkConnectionsGet/ConnectionsGet-1000-1000-4               367553        238289        -35.17%
BenchmarkConnectionsGet/ConnectionsGet-10000-0-4                 6828847       2844228       -58.35%
BenchmarkConnectionsGet/ConnectionsGet-10000-5000-4              5467207       3067539       -43.89%
BenchmarkConnectionsGet/ConnectionsGet-10000-10000-4             5477285       2876635       -47.48%

benchmark                                                        old allocs     new allocs     delta
BenchmarkStoreClosedConnection/StoreClosedConnection-100-4       300            0              -100.00%
BenchmarkStoreClosedConnection/StoreClosedConnection-1000-4      3000           0              -100.00%
BenchmarkStoreClosedConnection/StoreClosedConnection-10000-4     30020          10             -99.97%
BenchmarkStoreClosedConnection/StoreClosedConnection-30000-4     90155          155            -99.83%
BenchmarkConnectionsGet/ConnectionsGet-100-0-4                   412            105            -74.51%
BenchmarkConnectionsGet/ConnectionsGet-100-50-4                  412            105            -74.51%
BenchmarkConnectionsGet/ConnectionsGet-100-100-4                 412            105            -74.51%
BenchmarkConnectionsGet/ConnectionsGet-1000-0-4                  4015           1005           -74.97%
BenchmarkConnectionsGet/ConnectionsGet-1000-500-4                4015           1005           -74.97%
BenchmarkConnectionsGet/ConnectionsGet-1000-1000-4               4015           1005           -74.97%
BenchmarkConnectionsGet/ConnectionsGet-10000-0-4                 40059          10025          -74.97%
BenchmarkConnectionsGet/ConnectionsGet-10000-5000-4              40076          10025          -74.99%
BenchmarkConnectionsGet/ConnectionsGet-10000-10000-4             40059          10025          -74.97%

benchmark                                                        old bytes     new bytes     delta
BenchmarkStoreClosedConnection/StoreClosedConnection-100-4       2400          0             -100.00%
BenchmarkStoreClosedConnection/StoreClosedConnection-1000-4      24122         60            -99.75%
BenchmarkStoreClosedConnection/StoreClosedConnection-10000-4     249774        4911          -98.03%
BenchmarkStoreClosedConnection/StoreClosedConnection-30000-4     815833        95966         -88.24%
BenchmarkConnectionsGet/ConnectionsGet-100-0-4                   41981         18365         -56.25%
BenchmarkConnectionsGet/ConnectionsGet-100-50-4                  41981         18366         -56.25%
BenchmarkConnectionsGet/ConnectionsGet-100-100-4                 41981         18365         -56.25%
BenchmarkConnectionsGet/ConnectionsGet-1000-0-4                  374974        195999        -47.73%
BenchmarkConnectionsGet/ConnectionsGet-1000-500-4                375050        196010        -47.74%
BenchmarkConnectionsGet/ConnectionsGet-1000-1000-4               375206        196020        -47.76%
BenchmarkConnectionsGet/ConnectionsGet-10000-0-4                 6912588       1821855       -73.64%
BenchmarkConnectionsGet/ConnectionsGet-10000-5000-4              6922656       1822900       -73.67%
BenchmarkConnectionsGet/ConnectionsGet-10000-10000-4             6932816       1823930       -73.69%

Nice! 🎉