search

aspnet / DataAccessPerformance

Benchmarks, prototypes and discussions for database access performance investigation
MIT License
115 stars 23 forks source link

Help understanding poor "Data updates" performance #49

Open bgrainger opened 6 years ago

bgrainger commented 6 years ago

The .NET Core entrants have very poor performance on the TechEmpower "Data updates" benchmark. For example, in the latest nightly run (filtered to MySQL), aspcore-ado-my (which is normally the fastest .NET MySQL entrant) is the second-slowest .NET result, and only 39.7% the speed of the top performer, fasthttp. These results have been fairly consistent across the TFB runs I've checked.

However, if I run the benchmarks locally (Windows 10 x64, using Docker Linux containers), I can't replicate these results; I get 1707 rps for fasthttp and 1830 rps for aspcore-ado-my.

Maybe I just don't have enough hardware to throw at the problem, so my results are artificially limited and with more cores, fasthttp would pull ahead? The official benchmarks say 28 threads and 512 connections but I just have 12 threads and 512 connections in the output below.

Finally, I don't understand why aspcore-ado-my has one of the slowest .NET Core MySQL "Data updates" results, but frameworks with more overhead, such as aspcore-mvc-dap-my, show more requests per second. Again, this is not something I can reproduce locally; aspcore-ado-my is the fastest performer out of all the .NET Core MySQL entrants. (Again, possibly something that only manifests in high-concurrency scenarios?)

Any thoughts?

fasthttp

Running 15s test @ http://tfb-server:8080/update?queries=20
  12 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   301.55ms  174.63ms   2.29s    80.20%
    Req/Sec   143.86     33.23   290.00     66.26%
  Latency Distribution
     50%  252.54ms
     75%  368.59ms
     90%  526.87ms
     99%  927.05ms
  25774 requests in 15.10s, 18.65MB read
Requests/sec:   1706.98
Transfer/sec:      1.23MB
STARTTIME 1535491268
ENDTIME 1535491283

aspcore-ado-my

Running 15s test @ http://tfb-server:8080/updates/queries=20
  12 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   275.97ms  147.95ms   1.02s    68.36%
    Req/Sec   153.44     51.48   350.00     68.13%
  Latency Distribution
     50%  261.33ms
     75%  366.65ms
     90%  475.94ms
     99%  685.26ms
  27505 requests in 15.03s, 19.98MB read
Requests/sec:   1830.47
Transfer/sec:      1.33MB
STARTTIME 1535492089
ENDTIME 1535492104
divega commented 6 years ago

@sebastienros any clues?

sebastienros commented 6 years ago

I haven't worked at all on this scenario so far, but that's on my priorities list. Last time Ben tried to make it fast, it actually didn't help, and the results where so unexpected we just submitted the changes without knowing what results we would get. We know it's the next major thing to fix. And not just for mysql.