search

baidu / dperf

dperf is a 100Gbps network load tester.
https://dperf.org
Apache License 2.0
4.93k stars 509 forks source link

设置一些参数配置后client必定会回一个ack #442

Closed alongpan closed 4 months ago

alongpan commented 4 months ago

当我设置keepalive大于2s的，或者是keepalive后面配置了num，或者配置protocol为http，client必定会确认一个ack，如果这些不配置client和server是一个请求一个响应，请问这个client的ack可以取消吗？ image

pengjianzhang commented 4 months ago

请详细说明一下不ACK的理由。 你试一下早期的版本，可能不会回复ACK。 我看下能不能给个配置参数关闭ACK

alongpan commented 4 months ago

有几种场景，如果不是测试其他服务器，只是dperf client和服务器对发数据 1、client回应ack会导致数据包多3/1（主要测pps的时候，这个倒是可以写公式解决，如果能参数控制回不回ack更好） 2、固定包长的时候回的ack是64字节，这个没法更改，比如固定测试128字节数据，出现1/3的64字节数据包

alongpan commented 4 months ago

这是我的配置，试了几次没有重传

daemon

mode client socket_mem 12000 tx_burst 8 launch_num 8 cpu 0 2 4 6 rss auto l3l4

payload_random

vlan 1234

packet_size 56 duration 3000h cps 40000 cc 1000000 keepalive 1320ms

port 0000:04:00.0 192.168.31.10 192.168.99.240 client 192.168.31.10 200 server 192.168.99.240 1 listen 80 2 lport_range 63026-65535 http_method POST

daemon

mode server socket_mem 12000 tx_burst 8 payload_random cpu 12 14 16 18 rss auto l3l4 vlan 1234

duration 3000h packet_size 56 keepalive 10s

port 0000:04:00.1 192.168.99.240 192.168.31.10 client 192.168.31.10 200 server 192.168.99.240 1 listen 80 2

pengjianzhang commented 4 months ago

http协议回复ACK太快了，应该是一个Bug keepalive超过2秒，dperf会快速回复ACK，可以做一个开关关掉

alongpan commented 4 months ago

我这个配置没超过2s，看数据包是不回ack的。 麻烦在基于post那个版本做个开关，谢谢。

pengjianzhang commented 4 months ago

http post已经合入到主干了 请在周围小伙伴里面宣传一下dperf 谢谢

pengjianzhang commented 4 months ago

请测一下这个分支 https://github.com/pengjianzhang/dperf/tree/disable_ack client上配置： disable_ack

alongpan commented 4 months ago

arpRx /home/dperf/build/dperf: unrecognized option '--no-telemetry' EAL: Detected 24 lcore(s) EAL: Detected 2 NUMA nodes EAL: FATAL: Invalid 'command line' arguments. EAL: Invalid 'command line' arguments.

Usage: /home/dperf/build/dperf [options]

EAL common options: -c COREMASK Hexadecimal bitmask of cores to run on -l CORELIST List of cores to run on The argument format is [-c2][,c3[-c4],...] where c1, c2, etc are core indexes between 0 and 128 --lcores COREMAP Map lcore set to physical cpu set The argument format is 'lcores[@cpus][,lcores[@cpus]...]' lcores and cpus list are grouped by '(' and ')' Within the group, '-' is used for range separator, ',' is used for single number separator. '( )' can be omitted for single element group, '@' can be omitted if cpus and lcores have the same value -s SERVICE COREMASK Hexadecimal bitmask of cores to be used as service cores --master-lcore ID Core ID that is used as master --mbuf-pool-ops-name Pool ops name for mbuf to use -n CHANNELS Number of memory channels -m MB Memory to allocate (see also --socket-mem) -r RANKS Force number of memory ranks (don't detect) -b, --pci-blacklist Add a PCI device in black list. Prevent EAL from using this PCI device. The argument format is . -w, --pci-whitelist Add a PCI device in white list. Only use the specified PCI devices. The argument format is <[domain:]bus:devid.func>. This option can be present several times (once per device). [NOTE: PCI whitelist cannot be used with -b option] --vdev Add a virtual device. The argument format is [,key=val,...] (ex: --vdev=net_pcap0,iface=eth2). --iova-mode Set IOVA mode. 'pa' for IOVA_PA 'va' for IOVA_VA -d LIB.so|DIR Add a driver or driver directory (can be used multiple times) --vmware-tsc-map Use VMware TSC map instead of native RDTSC --proc-type Type of this process (primary|secondary|auto) --syslog Set syslog facility --log-level= Set global log level --log-level=: Set specific log level -v Display version information on startup -h, --help This help --in-memory Operate entirely in memory. This will disable secondary process support --base-virtaddr Base virtual address

EAL options for DEBUG use only: --huge-unlink Unlink hugepage files after init --no-huge Use malloc instead of hugetlbfs --no-pci Disable PCI --no-hpet Disable HPET --no-shconf No shared config (mmap'd files)

EAL Linux options: --socket-mem Memory to allocate on sockets (comma separated values) --socket-limit Limit memory allocation on sockets (comma separated values) --huge-dir Directory where hugetlbfs is mounted --file-prefix Prefix for hugepage filenames --create-uio-dev Create /dev/uioX (usually done by hotplug) --vfio-intr Interrupt mode for VFIO (legacy|msi|msix) --legacy-mem Legacy memory mode (no dynamic allocation, contiguous segments) --single-file-segments Put all hugepage memory in single files --match-allocations Free hugepages exactly as allocated

rte_eal_init fail dpdk_eal_init fail dpdk init fail

alongpan commented 4 months ago

这个版本启动报错，一摸一样的配置回退就可以了 mode client socket_mem 12000 cpu 0 2 4 6 rss auto l3l4 tx_burst 8 launch_num 8 duration 3000h cps 40000 cc 2000010 vlan 1234 client 192.168.31.10 200 server 192.168.99.240 1 listen 80 2 http_method POST packet_size 128 keepalive 5980ms lport_range 60526-65535 port 0000:04:00.0 192.168.31.10 192.168.99.240

pengjianzhang commented 4 months ago

你用的是哪个dpdk版本

alongpan commented 4 months ago

19.11.10

pengjianzhang commented 4 months ago

修复了，更新一下代码，再试一下

alongpan commented 4 months ago

还是有问题，无法启动 Error: bad gateway. dperf cannot find gateway's MAC address. Please check the link.

pengjianzhang commented 4 months ago

port 0000:04:00.1 192.168.99.240 192.168.31.10 这两个IP在同一个网段吗 192.168.99.240 192.168.31.10 ？

alongpan commented 4 months ago

不是同一个网段，没过三层的，改成老版本可以跑通

pengjianzhang commented 4 months ago

192.168.31.10 是LB地址？ 如果不是同一网段，那就要配MAC地址 这不是BUG

alongpan commented 4 months ago

不是LB，我们这个配置不写掩码，实际下发配置会带掩码吗？我的环境非常简单，就是client和server对发，中间过二层交换机

alongpan commented 4 months ago

client和server都改成192.168.31.x也不行，用回老版本192.168.99.240 192.168.31.10也可以通

pengjianzhang commented 4 months ago

如果client server互打，那么配置就该这样

client A 200 server B 1 port 0000:04:00.0 A B

alongpan commented 4 months ago

是的，两边的ip和网关是互指的

pengjianzhang commented 4 months ago

你把client server都改成31网段，把两边配置发一下

alongpan commented 4 months ago

mode client socket_mem 12000 cpu 0 2 4 6 rss auto l3l4 tx_burst 8 launch_num 8 duration 3000h cps 40000 cc 2000010 vlan 1234 client 192.168.31.10 200 server 192.168.31.250 1 listen 80 2 packet_size 128 keepalive 5980ms lport_range 60526-65535 port 0000:04:00.0 192.168.31.10 192.168.31.250 http_method POST disable_ack

alongpan commented 4 months ago

mode server socket_mem 12000 tx_burst 8 payload_random cpu 12 14 16 18 rss auto l3l4 vlan 1234

duration 3000h packet_size 128 keepalive 10s

port 0000:04:00.1 192.168.31.250 192.168.31.10 client 192.168.31.10 200 server 192.168.31.250 1 listen 80 2

pengjianzhang commented 4 months ago
  1. 用disable_ack分支，这个配置会出现找不到网关MAC吗?
  2. 用哪个版本是没有错的？
alongpan commented 4 months ago

1、是的 2、我上一个版本是用的post那个分支

pengjianzhang commented 4 months ago

两边都同时启动看看，把最后的输出贴一下，主要是 看下arp统计 如： Total Numbers: ... ... arpRx 2 arpTx 2

pengjianzhang commented 4 months ago

另外，请试一下主干 是否有错 https://github.com/baidu/dperf/tree/main

alongpan commented 4 months ago

没看到arp统计，怎么看呢？ 感觉是启动太慢，要么client或者是server，启动接近20秒，一端已经结束了，两外一端还没开始。 image

alongpan commented 4 months ago

arpRx /home/dperf/build/dperf: unrecognized option '--no-telemetry' EAL: Detected 24 lcore(s) EAL: Detected 2 NUMA nodes EAL: FATAL: Invalid 'command line' arguments. EAL: Invalid 'command line' arguments.

Usage: /home/dperf/build/dperf [options]

EAL common options: -c COREMASK Hexadecimal bitmask of cores to run on -l CORELIST List of cores to run on The argument format is [-c2][,c3[-c4],...] where c1, c2, etc are core indexes between 0 and 128 --lcores COREMAP Map lcore set to physical cpu set The argument format is 'lcores[@CPUs][,lcores[@CPUs]...]' lcores and cpus list are grouped by '(' and ')' Within the group, '-' is used for range separator, ',' is used for single number separator. '( )' can be omitted for single element group, '@' can be omitted if cpus and lcores have the same value -s SERVICE COREMASK Hexadecimal bitmask of cores to be used as service cores --master-lcore ID Core ID that is used as master --mbuf-pool-ops-name Pool ops name for mbuf to use -n CHANNELS Number of memory channels -m MB Memory to allocate (see also --socket-mem) -r RANKS Force number of memory ranks (don't detect) -b, --pci-blacklist Add a PCI device in black list. Prevent EAL from using this PCI device. The argument format is domain:bus:devid.func. -w, --pci-whitelist Add a PCI device in white list. Only use the specified PCI devices. The argument format is <[domain:]bus:devid.func>. This option can be present several times (once per device). [NOTE: PCI whitelist cannot be used with -b option] --vdev Add a virtual device. The argument format is [,key=val,...] (ex: --vdev=net_pcap0,iface=eth2). --iova-mode Set IOVA mode. 'pa' for IOVA_PA 'va' for IOVA_VA -d LIB.so|DIR Add a driver or driver directory (can be used multiple times) --vmware-tsc-map Use VMware TSC map instead of native RDTSC --proc-type Type of this process (primary|secondary|auto) --syslog Set syslog facility --log-level= Set global log level --log-level=: Set specific log level -v Display version information on startup -h, --help This help --in-memory Operate entirely in memory. This will disable secondary process support --base-virtaddr Base virtual address

EAL options for DEBUG use only: --huge-unlink Unlink hugepage files after init --no-huge Use malloc instead of hugetlbfs --no-pci Disable PCI --no-hpet Disable HPET --no-shconf No shared config (mmap'd files)

EAL Linux options: --socket-mem Memory to allocate on sockets (comma separated values) --socket-limit Limit memory allocation on sockets (comma separated values) --huge-dir Directory where hugetlbfs is mounted --file-prefix Prefix for hugepage filenames --create-uio-dev Create /dev/uioX (usually done by hotplug) --vfio-intr Interrupt mode for VFIO (legacy|msi|msix) --legacy-mem Legacy memory mode (no dynamic allocation, contiguous segments) --single-file-segments Put all hugepage memory in single files --match-allocations Free hugepages exactly as allocated

rte_eal_init fail dpdk_eal_init fail dpdk init fail

主线也报这个错

pengjianzhang commented 4 months ago

给我邮箱发个邮件 pengjianzhang@gmail.com 会议讨论一下吧

pengjianzhang commented 4 months ago

再更新一下主干

pengjianzhang commented 4 months ago

请再试一下 https://github.com/pengjianzhang/dperf/tree/disable_ack

alongpan commented 4 months ago

还是有几个问题： 1、启动时间感觉太长，配置protocol 为HTTP也会提示找不到网关 2、配置disable_ack之后，服务器触发了重传，这个是否也能关掉 3、实际发出的数据包比配置的pcap_size多10byte（这个历史版本都一样）

alongpan commented 4 months ago

image

pengjianzhang commented 4 months ago

看邮件 会议讨论一下

pengjianzhang commented 4 months ago

请测试 https://github.com/pengjianzhang/dperf/tree/disable_ack

配置参考：

keepalive 6s retransmit_timeout 10 disable_ack

alongpan commented 4 months ago

可以了

pengjianzhang commented 4 months ago

这个特性已经合入了主干，请帮忙测试一下主干，近期会发布新版本，谢谢