Open kwjjyn opened 5 years ago
ajamshed
commented
5 years ago @kwjjyn,
Can you please post the following information:
1) The version of dpdk you are using,
2) The NIC adapter that you are using, and
3) The contents of the ./epserver.conf file.
Thanks.
kwjjyn
commented
5 years ago hi @ajamshed ，
1 the version of dpdk is 18.11.2 .
2 about the NIC adapter , the output of ethtool -i eth1 is :
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: 0000:00:03.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no3 the contents of the ./epserver.conf is :
############### mtcp configuration file ###############
# The underlying I/O module you want to use. Please
# enable only one out of the four.
#io = psio
#io = netmap
#io = onvm
io = dpdk
# No. of cores setting (enabling this option will override
# the `cpu' config for those applications that accept
# num_cores as command line arguments)
#
# e.g. in case ./epserver is executed with `-N 4', the
# mtcp core will still invoke 8 mTCP threads if the
# following line is uncommented.
num_cores = 8
# Core mask
core_mask = 000000FF
# Number of memory channels per processor socket (dpdk-only)
num_mem_ch = 4
#--- ONVM specific args ---#
# Service id (required)
#onvm_serv = 1
# Instance id (optional)
#onvm_inst = 1
# Destination id (will forward to another NF)
# If not set will send packets out
#onvm_dest = 2
# Sample ONVM configurations
# Single node epserver <-> epwget
#onvm_serv = 1
#onvm_dest = 2
# Simple endpoint server multi node setup
#onvm_serv = 1
#--------------------------#
# Enable multi-process support
#multiprocess = 1
# Used port (please adjust accordingly)
#------ PSIO ports -------#
#port = xge0 xge1
#port = xge1
#------ DPDK ports -------#
port = dpdk0
#port = dpdk1
#port = dpdk0 dpdk1
# Maximum concurrency per core (default = 10000)
#max_concurrency = 10000
# Maximum number of socket buffers per core (default = 10000)
# Set this to small value if there are many idle connections
#max_num_buffers = 10000
# Receive buffer size of sockets; if not set: rcvbuf = sndbuf
rcvbuf = 8192
# Send buffer size of sockets; if not set: sndbuf = rcvbuf
sndbuf = 8192
# if sndbuf & rcvbuf not set: sndbuf = rcvbuf = 8192
# TCP timeout seconds
# (tcp_timeout = -1 can disable the timeout check)
tcp_timeout = 30
# TCP timewait seconds
tcp_timewait = 0
# Interface to print stats (please adjust accordingly)
# You can enable multiple ports in separate lines
#------ PSIO ports -------#
#stat_print = xge0
#stat_print = xge1
#------ DPDK ports -------#
stat_print = dpdk0
#stat_print = dpdk1Another thing to mention is that ： I use testpmd to find out that the rx offload capabilities is below :
testpmd> show port 0 rx_offload capabilities
Rx Offloading Capabilities of port 0 :
Per Queue :
Per Port : VLAN_STRIP UDP_CKSUM TCP_CKSUM TCP_LRO VLAN_FILTERwhich means 0x21d , and I find the rx offload that mtcp required is below from the struct port_conf (111 lines):
.offloads = (
#if RTE_VERSION < RTE_VERSION_NUM(18, 5, 0, 0)
DEV_RX_OFFLOAD_CRC_STRIP |
#endif /* !18.05 */
DEV_RX_OFFLOAD_CHECKSUM
#ifdef ENABLELRO
| DEV_RX_OFFLOAD_TCP_LRO
#endif
),however, DEV_RX_OFFLOAD_CHECKSUM includes ipv4_checksum which is not included in rx offload capabilities .
so I comment DEV_RX_OFFLOAD_CHECKSUM in rx and tx offload . Then the error disappeared . But there is another error occured ：
[root@shangqing example]# ./epserver -p /root/mtcp/html/ -f epserver.conf
---------------------------------------------------------------------------------
Loading mtcp configuration from : epserver.conf
Loading interface setting
EAL: Detected 32 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: PCI device 0000:00:07.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 1af4:1000 net_virtio
Total number of attached devices: 1
Interface name: dpdk0
EAL: Auto-detected process type: PRIMARY
Configurations:
Number of CPU cores available: 8
Number of CPU cores to use: 8
Maximum number of concurrency per core: 10000
Maximum number of preallocated buffers per core: 10000
Receive buffer size: 8192
Send buffer size: 8192
TCP timeout seconds: 30
TCP timewait seconds: 0
NICs to print statistics: dpdk0
---------------------------------------------------------------------------------
Interfaces:
name: dpdk0, ifindex: 0, hwaddr: 00:16:3E:10:32:E7, ipaddr: 192.168.0.101, netmask: 255.255.255.255
Number of NIC queues: 8
---------------------------------------------------------------------------------
Loading routing configurations from : config/route.conf
fopen: No such file or directory
Skip loading static routing table
Routes:
Destination: 192.168.0.101/32, Mask: 255.255.255.255, Masked: 192.168.0.101, Route: ifdx-0
---------------------------------------------------------------------------------
Loading ARP table from : config/arp.conf
fopen: No such file or directory
Skip loading static ARP table
ARP Table:
(blank)
---------------------------------------------------------------------------------
Initializing port 0... done:
[dpdk_load_module: 750] Failed to get flow control info!
[dpdk_load_module: 757] Failed to set flow control info!: errno: -95
Port 0, MAC address: 00:16:3E:10:32:E7
Checking link statusdone
Port 0 Link Up - speed 10000 Mbps - full-duplex
third
[mtcp_destroy:1572] All MTCP threads are joined.So I think this is not the right way.
Another thing worth to be mentioned is that I deployed a vm from vultr.com to run mtcp and followed the install guide of dpdk step by step . There is the same problem.
Could you give me some advices ?
ajamshed
commented
5 years ago @kwjjyn :
Like I said previously, we do not provide support for virtio drivers. However, since you have already made significant progress with the patch-up work, I can try to help you out. It looks like one of the threads quits unexpectedly. Can you please try doing the following:
1- Use ./configure --with-dpdk-lib=$RTE_SDK/$RTE_TARGET --disable-hwcsum and -g gcc switch to re-compile mTCP source.
2- Run epserver on only 1 CPU.
3- Please check the dmesg log in case you see the same output.
hi , mtcp team when I run the epserver using the command:
something wrong happened with rte_eth_dev_configure() ,the output is :
Could you give me some advices? Actually there are 32 lcores. After I shut down the HT using the below command because the adjacent processor id has the same core id, there're 16 cpu cores ：
However , the left cpu ID is even number like cpu0 ,cpu2, cpu4...cpu30. Will this have an effect on the running results ? Actually the RSS RX queue is 8 , so I run the below command to configure the mtcp:
I would appreciate it if you could offer me any help .