F-Stack / f-stack

F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API.
http://www.f-stack.org
Other
3.8k stars 886 forks source link

ff_epoll_ctl() returns EINVAL error #617

Open shinikpark opened 2 years ago

shinikpark commented 2 years ago

I have no problem for setting up dpdk and compiling f-stack, neither. However, when I try to run example epoll source code, ff_epoll_ctl() function returns EINVAL error. My server OS: Ubuntu 18.04 / Kernel: Linux 5.11.

In the main() function of example/main_epoll.c:

int main(int argc, char * argv[])
{
    ff_init(argc, argv);

    sockfd = ff_socket(AF_INET, SOCK_STREAM, 0);
    printf("sockfd:%d\n", sockfd);
    if (sockfd < 0) {
        printf("ff_socket failed\n");
        exit(1);
    }

    int on = 1;
    ff_ioctl(sockfd, FIONBIO, &on);

    struct sockaddr_in my_addr;
    bzero(&my_addr, sizeof(my_addr));
    my_addr.sin_family = AF_INET;
    my_addr.sin_port = htons(80);
    my_addr.sin_addr.s_addr = htonl(INADDR_ANY);

    int ret = ff_bind(sockfd, (struct linux_sockaddr *)&my_addr, sizeof(my_addr));
    if (ret < 0) {
        printf("ff_bind failed\n");
        exit(1);
    }

    ret = ff_listen(sockfd, MAX_EVENTS);
    if (ret < 0) {
        printf("ff_listen failed\n");
        exit(1);
    }

    assert((epfd = ff_epoll_create(0)) > 0);
    ev.data.fd = sockfd;
    ev.events = EPOLLIN;

    if( ff_epoll_ctl(epfd, EPOLL_CTL_ADD, sockfd, &ev) != 0 ){
        printf("ff_epoll_ctl failed:%d, %s\n", errno,
                strerror(errno));
    }

    ff_run(loop, NULL);
    return 0;
}

When I runs the example/main_epoll.c, the output is described like this.

[dpdk]: lcore_mask=1
[dpdk]: channel=4
[dpdk]: promiscuous=1
[dpdk]: numa_on=1
[dpdk]: tx_csum_offoad_skip=0
[dpdk]: tso=0
[dpdk]: vlan_strip=1
[dpdk]: idle_sleep=0
[dpdk]: pkt_tx_delay=100
[dpdk]: symmetric_rss=0
[dpdk]: allow=18:00.1
[dpdk]: port_list=0
[dpdk]: nb_vdev=0
[dpdk]: nb_bond=0
[port0]: addr=10.10.10.1
[port0]: netmask=255.255.225.0
[port0]: broadcast=10.10.10.255
[port0]: gateway=10.10.10.0
[kni]: enable=1
[kni]: method=reject
[kni]: tcp_port=80,443,8080,8081
[kni]: udp_port=53
[freebsd.boot]: hz=100
[freebsd.boot]: fd_reserve=512
[freebsd.boot]: kern.ipc.maxsockets=262144
[freebsd.boot]: net.inet.tcp.syncache.hashsize=4096
[freebsd.boot]: net.inet.tcp.syncache.bucketlimit=100
[freebsd.boot]: net.inet.tcp.tcbhashsize=65536
[freebsd.boot]: kern.ncallout=262144
[freebsd.boot]: kern.features.inet6=1
[freebsd.boot]: net.inet6.ip6.auto_linklocal=1
[freebsd.boot]: net.inet6.ip6.accept_rtadv=2
[freebsd.boot]: net.inet6.icmp6.rediraccept=1
[freebsd.boot]: net.inet6.ip6.forwarding=0
[freebsd.sysctl]: kern.ipc.somaxconn=32768
[freebsd.sysctl]: kern.ipc.maxsockbuf=16777216
[freebsd.sysctl]: net.link.ether.inet.maxhold=5
[freebsd.sysctl]: net.inet.tcp.fast_finwait2_recycle=1
[freebsd.sysctl]: net.inet.tcp.sendspace=1638400
[freebsd.sysctl]: net.inet.tcp.recvspace=819200
[freebsd.sysctl]: net.inet.tcp.cc.algorithm=cubic
[freebsd.sysctl]: net.inet.tcp.sendbuf_max=16777216
[freebsd.sysctl]: net.inet.tcp.recvbuf_max=16777216
[freebsd.sysctl]: net.inet.tcp.sendbuf_auto=1
[freebsd.sysctl]: net.inet.tcp.recvbuf_auto=1
[freebsd.sysctl]: net.inet.tcp.sendbuf_inc=16384
[freebsd.sysctl]: net.inet.tcp.sack.enable=0
[freebsd.sysctl]: net.inet.tcp.blackhole=1
[freebsd.sysctl]: net.inet.tcp.msl=2000
[freebsd.sysctl]: net.inet.tcp.delayed_ack=0
[freebsd.sysctl]: net.inet.udp.blackhole=1
[freebsd.sysctl]: net.inet.ip.redirect=0
[freebsd.sysctl]: net.inet.ip.forwarding=0
[freebsd.sysctl]: net.inet.tcp.functions_default=freebsd
f-stack -c1 -n4 --proc-type=primary EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: Probe PCI driver: net_ice (8086:1592) device: 0000:18:00.1 (socket 0)
ice_load_pkg_type(): Active package is: 1.3.30.0, ICE COMMS Package
EAL: No legacy callbacks, legacy socket not created
lcore: 0, port: 0, queue: 0
create mbuf pool on socket 0
create ring:dispatch_ring_p0_q0 success, 2047 ring entries are now free!
create kni on port 0 success!
create kni ring success, 2047 ring entries are now free!
Port 0 MAC: 40 a6 b7 18 fd a1
Port 0 modified RSS hash function based on hardware support,requested:0x3ffffc configured:0x7ffc
RX checksum offload supported
TX ip checksum offload supported
TX TCP&UDP checksum offload supported
TSO is disabled
port[0]: rss table size: 512
set port 0 to promiscuous mode ok

Checking link statusdone
Port 0 Link Up - speed 100000 Mbps - full-duplex
link_elf_lookup_symbol: missing symbol hash table
link_elf_lookup_symbol: missing symbol hash table
Timecounters tick every 10.000 msec
WARNING: Adding ifaddrs to all fibs has been turned off by default. Consider tuning net.add_addr_allfibs if needed
Attempting to load tcp_bbr
tcp_bbr is now available
TCP Hpts created 1 swi interrupt threads and bound 0 to cpus
Timecounter "ff_clock" frequency 100 Hz quality 1
TCP_ratelimit: Is now initialized
f-stack-0: No addr6 config found.
f-stack-0: Ethernet address: 40:a6:b7:18:fd:a1
sockfd:512
ff_epoll_ctl failed:22, Invalid argument
Configure network interface of 0 up

I would like to know how I can fix this issues.

freak82 commented 2 years ago

Can you post the build output also? It may be helpful to see the build flags used for building the example application.

shinikpark commented 2 years ago

This is build output of example.

cc -O -gdwarf-2 -include rte_config.h -march=native -I/usr/local/include -o "helloworld"_epoll main_epoll.c -L/usr/local/lib/x86_64-linux-gnu -Wl,--whole-archive -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_qat.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_tap.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a -l:librte_crypto_bcmfs.a -l:librte_crypto_caam_jr.a -l:librte_crypto_dpaa_sec.a -l:librte_crypto_dpaa2_sec.a -l:librte_crypto_nitrox.a -l:librte_crypto_null.a -l:librte_crypto_octeontx.a -l:librte_crypto_octeontx2.a -l:librte_crypto_scheduler.a -l:librte_crypto_virtio.a -l:librte_compress_octeontx.a -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_event_dlb.a -l:librte_event_dlb2.a -l:librte_event_dpaa.a -l:librte_event_dpaa2.a -l:librte_event_octeontx2.a -l:librte_event_opdl.a -l:librte_event_skeleton.a -l:librte_event_sw.a -l:librte_event_dsw.a -l:librte_event_octeontx.a -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -L/home/intel/f-stack/lib -Wl,--whole-archive,-lfstack,--no-whole-archive -Wl,--no-whole-archive -lrt -lm -ldl -lcrypto -pthread -lnuma

jfb8856606 commented 2 years ago

You can git pull the lastest commit and try it again.

anqiangli commented 2 years ago

f-stack tcp can not create a real connection with client #623 Please response the issues #623 ,thanks!