iqiyi / dpvs

DPVS is a high performance Layer-4 load balancer based on DPDK.
Other
3k stars 723 forks source link

求助,测试dpvs时,cps上不去 #876

Closed jesson1 closed 1 year ago

jesson1 commented 1 year ago

您好,想请教一个问题

我在用dperf测试dpvs的cps,单臂模式,测试拓扑图如下: image 本意是打算慢慢调高dperf客户端的cps配置,看看dpvs处理连接的能力,但客户端cps仅仅只有1k的时候,就出现了skErr,而且观察发现skCon一直降不下来。

客户端输出:

seconds 108                cpuUsage 0   0   0   0   0   0   0   0
pktRx   1,477              pktTx    3,475              bitsRx   944,976            bitsTx  1,936,144          dropTx  0
tcpRx   1,476              tcpTx    3,475              udpRx    0                  udpTx   0
arpRx   1                  arpTx    0                  icmpRx   0                  icmpTx  0
tosRx   0                  otherRx  0                  badRx    0
synRx   492                synTx    2,491              finRx    492                finTx   492                rstRx   0          rstTx 0
synRt   1,531              finRt    0                  ackRt    0                  pushRt  0
tcpDrop 0                  udpDrop  0
skOpen  960                skClose  976                skCon    3,883              skErr   484                rtt(us) 848.4
httpGet 492                http2XX  492                httpErr  0
ierrors 0                  oerrors  0                  imissed  0

服务端输出:

seconds 183                cpuUsage 0   0   0   0   0   0   0   0
pktRx   2,899              pktTx    4,083              bitsRx   1,975,616          bitsTx  2,438,592          dropTx  0
tcpRx   2,899              tcpTx    4,083              udpRx    0                  udpTx   0
arpRx   0                  arpTx    0                  icmpRx   0                  icmpTx  0
tosRx   0                  otherRx  0                  badRx    0
synRx   985                synTx    2,169              finRx    957                finTx   957                rstRx   0          rstTx 0
synRt   1,209              finRt    0                  ackRt    0                  pushRt  0
tcpDrop 0                  udpDrop  0
skOpen  960                skClose  1,718              skCon    2,750              skErr   761
httpGet 957                http2XX  957                httpErr  0
ierrors 0                  oerrors  0                  imissed  0

客户端配置

# cps client
mode            client
tx_burst        128
launch_num      10
cpu             0-7
rss auto mq_rx_none

payload_size    1
duration        2m
cps             1k

#port           pci             addr      gateway        [mac]
port            0000:00:05.0    10.1.0.6 10.1.0.50  52:54:00:92:4e:67

#               addr_start      num
client          10.1.0.6    1

#               addr_start      num
server          10.1.0.50       1
#               port_start      num
listen          80              1

服务端配置:

# cps server
#daemon
mode            server
tx_burst        128
cpu             0-7
rss auto mq_rx_none

duration        300m
payload_size    1

#numa2
port            0000:00:05.0    10.1.0.5    10.1.0.7  52:54:00:92:4e:67

#               addr_start      num
client          10.1.0.7       1

#               addr_start      num
server          10.1.0.5       1

#               port_start      num
listen          80              1

dpvs配置:

[root@localhost bin]# ./ipvsadm -ln
IP Virtual Server version 0.0.0 (size=0)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.1.0.50:80 rr
  -> 10.1.0.5:80                  FullNat 1      0          0
[root@localhost bin]# ./ipvsadm -G
VIP:VPORT            TOTAL    SNAT_IP              CONFLICTS  CONNS
10.1.0.50:80         1
                              10.1.0.7             0          0
jesson1 commented 1 year ago

我的网卡是不支持flow和rss的 我参考了这里https://github.com/iqiyi/dpvs/blob/master/doc/tutorial.md#Ubuntu16.04 将dpvs调成单个worker后看起来就正常很多,但这样一来我就没法利用多核了

我不太理解为什么会这个样子,网卡不支持rss,就只能单核跑dpvs嘛?

ywc689 commented 1 year ago

性能测试可以参考:https://github.com/iqiyi/dpvs/blob/master/test/release/v1.9.2/performance.md

不支持RSS的网卡,一般只一个核有流量,无法利用DPVS多核并行处理的能力。