search

erpc-io / eRPC

Efficient RPCs for datacenter networks
https://erpc.io/
Other
840 stars 138 forks source link

ibv related error when compile #3

Closed sylvia531 closed 6 years ago

sylvia531 commented 6 years ago

I am trying to run the project and meet several problems.

  1. I saw the Hello is move out. However, I still need to add the autorun_app_file with any app inorder to create the makefile form CMakeLink.txt.
  2. I run the make -j after create the make with cmake . -DPERF=OFF -DTRANSPORT=infiniband. However, I met a set of ibv related error shown as following:

[ 2%] Building CXX object CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport.cc.o [ 2%] Building CXX object CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport_datapath.cc.o /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport_datapath.cc: In member function ‘void erpc::RawTransport::post_recvs(size_t)’: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport_datapath.cc:203:24: error: invalid use of incomplete type ‘struct erpc::ibv_exp_wq_family’ int ret = wq_family->recv_burst(wq, &mp_recv_sge[mp_sge_idx], 1); ^~ In file included from /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport_datapath.cc:1:0: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.h:200:10: note: forward declaration of ‘struct erpc::ibv_exp_wq_family’ struct ibv_exp_wq_family wq_family; ^~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In destructor ‘erpc::RawTransport::~RawTransport()’: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:53:40: error: aggregate ‘erpc::RawTransport::~RawTransport()::ibv_exp_release_intf_params rel_intf_params’ has incomplete type and cannot be defined struct ibv_exp_release_intf_params rel_intf_params; ^~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:56:73: error: ‘ibv_exp_release_intf’ was not declared in this scope ibv_exp_release_intf(resolve.ib_ctx, wq_family, &rel_intf_params) == 0, ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:65:54: error: ‘ibv_exp_destroy_rwq_ind_table’ was not declared in this scope exit_assert(ibv_exp_destroy_rwq_ind_table(ind_tbl) == 0, ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:68:38: error: ‘ibv_exp_destroy_wq’ was not declared in this scope exit_assert(ibv_exp_destroy_wq(wq) == 0, "Failed to destroy WQ"); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_mp_recv_qp()’: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:260:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_wq_init_attr wq_init_attr’ has incomplete type and cannot be defined struct ibv_exp_wq_init_attr wq_init_attr; ^~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:263:26: error: ‘IBV_EXP_WQT_RQ’ was not declared in this scope wq_init_attr.wq_type = IBV_EXP_WQT_RQ; ^~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:269:29: error: ‘IBV_EXP_CREATE_WQ_MP_RQ’ was not declared in this scope wq_init_attr.comp_mask |= IBV_EXP_CREATE_WQ_MP_RQ; ^~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:270:34: error: ‘IBV_EXP_MP_RQ_NO_SHIFT’ was not declared in this scope wq_init_attr.mp_rq.use_shift = IBV_EXP_MP_RQ_NO_SHIFT; ^~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:273:55: error: ‘ibv_exp_create_wq’ was not declared in this scope wq = ibv_exp_create_wq(resolve.ib_ctx, &wq_init_attr); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:277:26: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_wq_attr wq_attr’ has incomplete type and cannot be defined struct ibv_exp_wq_attr wq_attr; ^~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:279:23: error: ‘IBV_EXP_WQ_ATTR_STATE’ was not declared in this scope wq_attr.attr_mask = IBV_EXP_WQ_ATTR_STATE; ^~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:280:22: error: ‘IBV_EXP_WQS_RDY’ was not declared in this scope wq_attr.wq_state = IBV_EXP_WQS_RDY; ^~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:281:43: error: ‘ibv_exp_modify_wq’ was not declared in this scope rt_assert(ibv_exp_modify_wq(wq, &wq_attr) == 0, "Failed to ready WQ"); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:284:8: error: use of enum ‘ibv_exp_query_intf_status’ without previous declaration enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK; ^~~~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:284:48: error: ‘IBV_EXP_INTF_STAT_OK’ was not declared in this scope enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK; ^~~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:285:36: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_query_intf_params query_intf_params’ has incomplete type and cannot be defined struct ibv_exp_query_intf_params query_intf_params; ^~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:287:34: error: ‘IBV_EXP_INTF_GLOBAL’ was not declared in this scope query_intf_params.intf_scope = IBV_EXP_INTF_GLOBAL; ^~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:288:28: error: ‘IBV_EXP_INTF_WQ’ was not declared in this scope query_intf_params.intf = IBV_EXP_INTF_WQ; ^~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:291:74: error: ‘ibv_exp_query_intf’ was not declared in this scope ibv_exp_query_intf(resolve.ib_ctx, &query_intf_params, &intf_status)); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:295:42: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_rwq_ind_table_init_attr rwq_ind_table_init_attr’ has incomplete type and cannot be defined struct ibv_exp_rwq_ind_table_init_attr rwq_ind_table_init_attr; ^~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:302:76: error: ‘ibv_exp_create_rwq_ind_table’ was not declared in this scope ibv_exp_create_rwq_ind_table(resolve.ib_ctx, &rwq_ind_table_init_attr); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:314:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_rx_hash_conf rx_hash_conf’ has incomplete type and cannot be defined struct ibv_exp_rx_hash_conf rx_hash_conf; ^~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:316:35: error: ‘IBV_EXP_RX_HASH_FUNC_TOEPLITZ’ was not declared in this scope rx_hash_conf.rx_hash_function = IBV_EXP_RX_HASH_FUNC_TOEPLITZ; ^~~~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:319:38: error: ‘IBV_EXP_RX_HASH_DST_PORT_UDP’ was not declared in this scope rx_hash_conf.rx_hash_fields_mask = IBV_EXP_RX_HASH_DST_PORT_UDP; ^~~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:326:28: error: ‘IBV_EXP_QP_INIT_ATTR_RX_HASH’ was not declared in this scope IBV_EXP_QP_INIT_ATTR_RX_HASH; ^~~~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:327:16: error: ‘struct ibv_exp_qp_init_attr’ has no member named ‘rx_hash_conf’ qp_init_attr.rx_hash_conf = &rx_hash_conf; ^~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::install_flow_rule()’: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:347:14: error: ‘ibv_exp_flow_spec_ipv4_ext’ was not declared in this scope sizeof(ibv_exp_flow_spec_ipv4_ext) + sizeof(ibv_exp_flow_spec_tcp_udp); ^~~~~~ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:72: error: expected ‘>’ before ‘’ token auto spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext >(buf); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:72: error: expected ‘(’ before ‘’ token /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:73: error: expected primary-expression before ‘>’ token auto spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext >(buf); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:79: error: expected ‘)’ before ‘;’ token auto spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext *>(buf); ^ /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:373:21: error: ‘IBV_EXP_FLOW_SPEC_IPV4_EXT’ was not declared in this scope spec_ipv4->type = IBV_EXP_FLOW_SPEC_IPV4_EXT; ^~~~~~ At global scope: cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror] cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror] cc1plus: all warnings being treated as errors make[2]: [CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport_datapath.cc.o] Error 1 make[2]: Waiting for unfinished jobs.... /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_recvs(uint8_t)’: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:470:16: error: invalid use of incomplete type ‘struct erpc::ibv_exp_wq_family’ wq_family->recv_burst(wq, &mp_recv_sge[i], 1); ^~ In file included from /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:4:0: /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.h:200:10: note: forward declaration of ‘struct erpc::ibv_exp_wq_family’ struct ibv_exp_wq_family wq_family; ^~~~~ At global scope: cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror] cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror] cc1plus: all warnings being treated as errors make[2]: [CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport.cc.o] Error 1 make[1]: [CMakeFiles/erpc.dir/all] Error 2 make: [all] Error 2

I wonder what I did wrong during build and compile process. Thanks.

anujkaliaiitd commented 6 years ago

Thanks for the comments. Compilation now works without autorun_app_file.

The ibv errors are likely due to an old libibverbs installation. Please try with the latest Mellanox OFED.

kernel-z commented 5 years ago

Thanks for the comments. Compilation now works without autorun_app_file.

The ibv errors are likely due to an old libibverbs installation. Please try with the latest Mellanox OFED.

which version? I used the 4.2-0.1.2.0

anujkaliaiitd commented 5 years ago

Can you try Mellanox OFED 4.4 or newer? If you must use 4.2 for some reason, I can take a look. Version 4.2 isn't officially compatible with the new OS distros, so it requires more work for testing.