Closed panzhengyu closed 1 year ago
不正常,这样修改一下试试 client 192.168.3.0 -> client 192.168.3.1
如果喜欢dperf,请给项目加个star,谢谢。也请向周围小伙伴推荐一下
已经star,是在dperf server还是在dperf client上修改,这样修改的目的是什么啊
lvs使用的fullnat模式,目前lvs转发到dperf server的源Ip只有一个192.168.3.100。
可能是一个Bug,client, server上都修改, 192.168.3.0 不是合法IP,可能引起了一个bug
修改后,没有生效。 我的本意是分配了一个大段IP 192.168.0.0/21用来压测,网关是192.168.7.254。 对于,dperf client 使用IP 192.168.3.1- 192.168.3.50。 压测的lvs server ip是192.168.5.1。 dperf server使用IP 192.168.6.100-192.168.6.106。从lvs到derf server使用lvs的local IP 192.168.3.100。
你server上 这么配试试 client 192.168.3.100 1
还是dperf server直接对lvs发送过来的syn包回复reset。
目前我的dperf client配置是
mode client
socket_mem 2048
cpu 0
slow_start 60
tx_burst 128
launch_num 10
payload_size 1
duration 90s
protocol tcp
cps 100k
port 0000:af:00.0 192.168.0.30 192.168.7.254 30:0d:9e:42:5d:50
client 192.168.3.1 50
server 192.168.5.1 1
listen 80 1
我的dperf server配置是
mode server
socket_mem 2048
cpu 8-14
tx_burst 128
payload_size 1
duration 10m
port 0000:af:00.1 192.168.1.30 192.168.7.254 00:00:64:01:01:01
client 192.168.3.100 1
server 192.168.6.100 7
listen 80 1
CPS 设置100 看看 server只用一个CPU试试 是主干代码吗
cpu只使用1个的话,报错
./build/dperf -c test/http/server-cps.conf
Error: cpu num less than server ip num at 'client' mode
使用的是v1.4.0 tag的代码。
CPS 设置100 同样报错
贴一下抓包
tcpdump -nnee -i ens2f1 host 192.168.6.103
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens2f1, link-type EN10MB (Ethernet), capture size 262144 bytes
18:23:24.069574 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.9 > 192.168.6.103.80: Flags [S], seq 1554452105, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.069704 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.9: Flags [R.], seq 0, ack 1554452106, win 0, length 0
18:23:24.277823 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.14 > 192.168.6.103.80: Flags [S], seq 706016934, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.277880 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.19 > 192.168.6.103.80: Flags [S], seq 1368424217, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.277922 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.14: Flags [R.], seq 0, ack 706016935, win 0, length 0
18:23:24.434076 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.30 > 192.168.6.103.80: Flags [S], seq 653853081, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.434105 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.19: Flags [R.], seq 0, ack 1368424218, win 0, length 0
18:23:24.434203 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.30: Flags [R.], seq 0, ack 653853082, win 0, length 0
18:23:24.590317 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.39 > 192.168.6.103.80: Flags [S], seq 1438276, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.590376 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.36 > 192.168.6.103.80: Flags [S], seq 1317208719, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.590411 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.36: Flags [R.], seq 0, ack 1317208720, win 0, length 0
18:23:24.590411 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.39: Flags [R.], seq 0, ack 1438277, win 0, length 0
18:23:24.746570 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.46 > 192.168.6.103.80: Flags [S], seq 2056905421, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.746654 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.47 > 192.168.6.103.80: Flags [S], seq 105442140, win 58400, options [mss 1460,nop,wscale 13], length 0
18:23:24.746705 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.46: Flags [R.], seq 0, ack 2056905422, win 0, length 0
18:23:24.746712 3c:fd:fe:b1:c6:99 > 28:de:e5:7f:ea:21, ethertype IPv4 (0x0800), length 60: 192.168.6.103.80 > 192.168.3.100.47: Flags [R.], seq 0, ack 105442141, win 0, length 0
^C
16 packets captured
16 packets received by filter
0 packets dropped by kernel
28:de:e5:7f:ea:21是lvs 的mac address 3c:fd:fe:b1:c6:99 是网关192.168.7.254的mac address 192.168.6.103是dperf server的地址 192.168.3.100是lvs的地址 是在lvs节点抓的
server只用1个CPU, cpu 8 server 192.168.6.100 1
检查一下你网络里面是不是有重复IP 贴一下dperf server的输出截图,看看这个rst是不是dperf server发出的
没有IP冲突。 我没开启dperf server的时候, lvs只抓到arp包。没有响应,说明没有冲突。
tcpdump -nnee -i ens2f1 host 192.168.6.103
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens2f1, link-type EN10MB (Ethernet), capture size 262144 bytes
18:52:45.908549 28:de:e5:7f:ea:21 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.6.103 tell 192.168.3.100, length 28
18:52:46.911288 28:de:e5:7f:ea:21 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.6.103 tell 192.168.3.100, length 28
18:52:47.913287 28:de:e5:7f:ea:21 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.6.103 tell 192.168.3.100, length 28
18:52:48.971017 28:de:e5:7f:ea:21 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.6.103 tell 192.168.3.100, length 28
这是dperf server的输出截图。
-------------
Test Finished
-------------
pktRx 14,775,788 pktTx 13,195,419 bitsRx 7,303,646,096 bitsTx 5,801,801,168 dropTx 0
tcpRx 14,774,491 tcpTx 1,583,954 udpRx 0 udpTx 0
arpRx 557 arpTx 172 icmpRx 0 icmpTx 0
tosRx 0 otherRx 740 badRx 0
synRx 1,581,938 synTx 1,583,954 finRx 0 finTx 0 rstRx 1,581,260 rstTx 11,611,293
synRt 2,019 finRt 0 ackRt 0 pushRt 0
tcpDrop 0 udpDrop 0
skOpen 1,581,935 skClose 1,581,933 skCon 2 skErr 673
httpGet 0 http2XX 0 httpErr 0
ierrors 0 oerrors 0 imissed 0
估计是端口重用太快了 你测一下并发 cc 1000 cps 100
dperf server改成只有1个server后, dperf client客户端cc 1000 cps 100 keepalive 1s配置, lvs抓发到dperf server,只有syn包,dperf server 不回应。
19:19:42.318649 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.142 > 192.168.6.103.80: Flags [S], seq 693888891, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:42.318650 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.144 > 192.168.6.103.80: Flags [S], seq 133489566, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:42.318740 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.140 > 192.168.6.103.80: Flags [S], seq 1253238617, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:42.318842 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.138 > 192.168.6.103.80: Flags [S], seq 1813637943, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:43.318642 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.143 > 192.168.6.103.80: Flags [S], seq 1842348294, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:43.318648 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.145 > 192.168.6.103.80: Flags [S], seq 1767156026, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:43.318739 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.141 > 192.168.6.103.80: Flags [S], seq 497867500, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:43.318848 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.139 > 192.168.6.103.80: Flags [S], seq 2010706886, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:44.318652 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.144 > 192.168.6.103.80: Flags [S], seq 133489566, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:44.318656 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.146 > 192.168.6.103.80: Flags [S], seq 1963176393, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:44.318740 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.142 > 192.168.6.103.80: Flags [S], seq 693888891, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:44.318843 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.140 > 192.168.6.103.80: Flags [S], seq 1253238617, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:45.318651 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.145 > 192.168.6.103.80: Flags [S], seq 1767156026, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:45.318655 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.147 > 192.168.6.103.80: Flags [S], seq 1918690383, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:45.318739 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.143 > 192.168.6.103.80: Flags [S], seq 1842348294, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:45.318842 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.141 > 192.168.6.103.80: Flags [S], seq 497867500, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:46.318640 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.146 > 192.168.6.103.80: Flags [S], seq 1963176393, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:46.318650 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.148 > 192.168.6.103.80: Flags [S], seq 2114710750, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:46.318737 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.144 > 192.168.6.103.80: Flags [S], seq 133489566, win 58400, options [mss 1460,nop,wscale 13], length 0
19:19:46.318839 28:de:e5:7f:ea:21 > 3c:fd:fe:b1:c6:99, ethertype IPv4 (0x0800), length 62: 192.168.3.100.142 > 192.168.6.103.80: Flags [S], seq 693888891, win 58400, options [mss 1460,nop,wscale 13], length 0
@pengjianzhang 能指导一下我在那个地方断点debug dperf代码吗
server需要配置keepalive 2s
邮箱给我发个腾讯会议 一起看下
好的
可以了,感谢 @pengjianzhang 我把dperf client和dperf server的gateway网关地址配置错了。gatway 应该配成lvs的mac地址。我配成和dperf client、dperf server、lvs之外的一个网关地址了。网关不能转发包导致。
不客气~~ 能否把你的问题写成一个博客文章,最好能画图,把连接发给我,方便其他小伙伴参考! 另外,请内部宣传一下。谢谢~
好的
继续请教一下, 我的目前dperf server配置是
mode server
socket_mem 2048
cpu 2-3
tx_burst 128
payload_size 1
duration 10m
keepalive 2s
port 0000:af:00.1 192.168.6.103 192.168.7.253 28:de:e5:7f:ea:21
client 192.168.3.100 1
server 192.168.6.103 2
listen 80 1
转发到192.168.6.103的都能正常回复,转发到192.168.6.104的都回复reset。 server 192.168.6.103 2 配置,不就是代表配置两个IP,192.168.6.103和192.168.6.104吗? 为什么192.168.6.104都回复reset? 另外,port配置中的IP作用是什么,和server中的IP有什么区别?
server 192.168.6.103 2 配置,不就是代表配置两个IP,192.168.6.103和192.168.6.104吗? 是 为什么192.168.6.104都回复reset? 不知道
另外,port配置中的IP作用是什么,和server中的IP有什么区别? ARP寻址,监听,详见配置文档
reset 原因是socket_server_lookup 返回NULL,这个是为什么啊
static inline void tcp_server_process(struct work_space *ws, struct rte_mbuf *m)
{
struct iphdr *iph = mbuf_ip_hdr(m);
struct tcphdr *th = mbuf_tcp_hdr(m);
uint8_t flags = th->th_flags;
struct socket *sk = NULL;
sk = socket_server_lookup(&ws->socket_table, iph, th);
if (unlikely(sk == NULL)) {
if (ws->kni && work_space_is_local_addr(ws, m)) {
return kni_recv(ws, m);
}
MBUF_LOG(m, "drop-no-socket");
if (g_config.tcp_rst) {
tcp_reply_rst(ws, m);
} else {
net_stats_tcp_drop();
mbuf_free2(m);
}
return;
}
IP,端口不匹配
这个一般是为什么不匹配啊,跟什么有关系啊
static inline struct socket *socket_common_lookup(const struct socket_table *st, uint32_t client_ip, uint32_t server_ip
, uint16_t client_port, uint16_t server_port)
{
uint32_t client_low_2byte = ntohl(client_ip) & 0xffff;
struct socket_port_table *t = st->ht[client_low_2byte];
uint16_t client_port_host = ntohs(client_port);
uint16_t server_port_host = ntohs(server_port);
if (likely(t != NULL) && (server_ip == st->server_ip) && (client_port != 0) &&
(server_port_host >= st->port_min) && (server_port_host <= st->port_max)) {
return socket_port_table_get(st, t, client_port_host, server_port_host);
}
return NULL;
}
发现是这个函数 server_ip == st->server_ip 条件不满足,是为什么啊
hi, 我正在测试lvs性能,发现从lvs发到dperf server的syn包,dperf server回复reset,这个正常吗?
我的dperf client配置是
我的dperf server配置是