search

coredns / perf-tests

Scripts and utilities for scale and performance testing of CoreDNS.
Apache License 2.0
17 stars 9 forks source link

Hit 100000 qps #4

Closed miekg closed 7 years ago

miekg commented 7 years ago 
% make LIMIT=40000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 40000 -n 100000 -o test.json
[05/01/17 09:21:25] Starting...
[05/01/17 09:21:28] Messages:
    Sent....: 100000;
    Received: 100000;
    Lost....: 0.

[05/01/17 09:21:28] Exiting...
% make LIMIT=50000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 50000 -n 100000 -o test.json
[05/01/17 09:21:36] Starting...
[05/01/17 09:21:38] Messages:
    Sent....: 100000;
    Received: 100000;
    Lost....: 0.

[05/01/17 09:21:38] Exiting...
% make LIMIT=60000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 60000 -n 100000 -o test.json
[05/01/17 09:21:41] Starting...
[05/01/17 09:21:53] Messages:
    Sent....: 100000;
    Received: 99934;
    Lost....: 66.

[05/01/17 09:21:53] Exiting...
% make LIMIT=60000 mig                                           12s ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 60000 -n 100000 -o test.json
[05/01/17 09:21:56] Starting...
[05/01/17 09:21:58] Messages:
    Sent....: 100000;
    Received: 100000;
    Lost....: 0.

[05/01/17 09:21:58] Exiting...
% make LIMIT=70000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 70000 -n 100000 -o test.json
[05/01/17 09:22:02] Starting...
[05/01/17 09:22:14] Messages:
    Sent....: 100000;
    Received: 99483;
    Lost....: 517.

[05/01/17 09:22:14] Exiting...
% make LIMIT=70000 mig                                           12s ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 70000 -n 100000 -o test.json
[05/01/17 09:22:15] Starting...
[05/01/17 09:22:16] Messages:
    Sent....: 100000;
    Received: 100000;
    Lost....: 0.

[05/01/17 09:22:16] Exiting...
% make LIMIT=80000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 80000 -n 100000 -o test.json
[05/01/17 09:22:21] Starting...
[05/01/17 09:22:22] Messages:
    Sent....: 100000;
    Received: 100000;
    Lost....: 0.

[05/01/17 09:22:22] Exiting...
% make LIMIT=90000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 90000 -n 100000 -o test.json
[05/01/17 09:22:26] Starting...
[05/01/17 09:22:37] Messages:
    Sent....: 100000;
    Received: 99920;
    Lost....: 80.

[05/01/17 09:22:37] Exiting...
% make LIMIT=100000 mig                                                         11s ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 100000 -n 100000 -o test.json
[05/01/17 09:22:42] Starting...
[05/01/17 09:22:43] Messages:
    Sent....: 100000;
    Received: 100000;
    Lost....: 0.

[05/01/17 09:22:43] Exiting...
miekg commented 7 years ago

Needs this patch for Go DNS: https://gist.github.com/miekg/e80d2eadfea48422178cdbaaf9f8a33f

miekg commented 7 years ago

queryperf gets to 140000:

% make TIME=3 queryperf                                                             ~/git/coredns/perf-tests/local-perf master
./bin/queryperf-linux_amd64 -d domain.lst.queryperf -l 3 -s 127.0.0.1 -p 1053

DNS Query Performance Testing Tool
Version: $Id: queryperf.c,v 1.12 2007/09/05 07:36:04 marka Exp $

[Status] Processing input data
[Status] Sending queries (beginning with 127.0.0.1)
[Status] Testing complete

Statistics:

  Parse input file:     multiple times
  Run time limit:       3 seconds
  Ran through file:     143281 times

  Queries sent:         429845 queries
  Queries completed:    429845 queries
  Queries lost:         0 queries
  Queries delayed(?):   0 queries

  RTT max:          0.008106 sec
  RTT min:              0.000004 sec
  RTT average:          0.000123 sec
  RTT std deviation:    0.000156 sec
  RTT out of range:     0 queries

  Percentage completed: 100.00%
  Percentage lost:        0.00%

  Started at:           Mon May  1 09:33:15 2017
  Finished at:          Mon May  1 09:33:18 2017
  Ran for:              3.000063 seconds

  Queries per second:   143278.657815 qps
miekg commented 7 years ago

See this repo for someone added this to Go socket handling. Big thing is to do it portable, don't why upstream Go does not do this.

miekg commented 7 years ago

I'm going to port/fix various things in Go DNS and see how portable things will be.

miekg commented 7 years ago

Now I'm loosing my sanity because a downloaded CoreDNS-006 (from the releases) also shows this speedup... Trying to arrive at those 45K again, just to double check...

miekg commented 7 years ago

Utterly confused :-( https://github.com/miekg/exdns/tree/master/reflect does 170,000 qps on my machine, with, or, without a patched go DNS

miekg commented 7 years ago

Thought about, but I should also have checked in the CoreDNS binary... :/

haosdent commented 5 years ago

@miekg sry, I have enabled reuseport according to your PR at https://github.com/coredns/coredns/pull/2149 . But still could not reach 100k QPS.

Is there any special config could archieve this?