zrlio / softiwarp

SoftiWARP: Software iWARP kernel driver and user library for Linux
129 stars 47 forks source link

softiwarp does not build on Ubuntu #24

Open Luo-Liang opened 6 years ago

Luo-Liang commented 6 years ago

Hi I am building softiwarp from Ubuntu with kernel 4.4.0-1062-aws, but the kernel folder wouldn't compile and this is the error:

/home/ubuntu/softiwarp/kernel/siw_mem.c:163:9: error: too few arguments to function ‘get_user_pages’
    rv = get_user_pages(first_page_va, nents, 1, 1, plist,
In file included from include/linux/scatterlist.h:7:0,
                 from /home/ubuntu/softiwarp/kernel/siw_mem.c:40:

Any idea how i can deal with this?

Thanks!

Luo-Liang commented 6 years ago

I have changed the line to

rv = get_user_pages(current, current->mm,first_page_va, nents, 1, 1, plist, NULL);

and it seems like it builds. Let me know if that's incorrect. Thanks!

Luo-Liang commented 6 years ago

<infiniband/driver.h> not found. Is libibverbs installed? Now when running autogen, I'm seeing this :/

mselseth commented 6 years ago

Having similar build issue. Using ubuntu kernel 4.15.0-32-generic, release 18.04. Attempted to build userlib code ./configure fails with: +++ checking for ibv_get_device_list in -libibverbs.so... no configure: error: ibv_get_device_list() not found. libsiw requires libibverbs.so.

To get to this point I modified configure.in (configure.ac) to resolve libibverbs.so correctly. In configure.ac it was: AC_CHECK_LIB(ibverbs.so, ibv_get_device_list, [], changed it to: AC_CHECK_LIB(ibibverbs.so, ibv_get_device_list, [],

If ignoring this check failure, the next failure is similar to above: +++ checking for ibv_get_device_list in -libibverbs.so... no configure: WARNING: ibv_get_device_list() not found. libsiw requires libibverbs.so. checking sysfs/libsysfs.h usability... no checking sysfs/libsysfs.h presence... no checking for sysfs/libsysfs.h... no checking infiniband/driver.h usability... no checking infiniband/driver.h presence... no checking for infiniband/driver.h... no configure: error: <infiniband/driver.h> not found. Is libibverbs installed?

mselseth commented 6 years ago

Confirmed the function exists in the library. ++++ objdump -d ./usr/lib/x86_64-linux-gnu/libibverbs.so | grep ibv_get_device_list 000000000000a280 ibv_get_device_list@IBVERBS_1.0: a2a2: e8 59 09 00 00 callq ac00 ibv_get_device_list@@IBVERBS_1.1 a2aa: 0f 84 d0 00 00 00 je a380 ibv_get_device_list@IBVERBS_1.0+0x100 a2cd: 74 7c je a34b ibv_get_device_list@IBVERBS_1.0+0xcb a2d8: 0f 8e 92 00 00 00 jle a370 ibv_get_device_list@IBVERBS_1.0+0xf0 a2e3: eb 1a jmp a2ff ibv_get_device_list@IBVERBS_1.0+0x7f a2fd: 7e 71 jle a370 ibv_get_device_list@IBVERBS_1.0+0xf0 a316: 75 d0 jne a2e8 ibv_get_device_list@IBVERBS_1.0+0x68 a323: 7e 1e jle a343 ibv_get_device_list@IBVERBS_1.0+0xc3 a330: 74 05 je a337 ibv_get_device_list@IBVERBS_1.0+0xb7 a341: 7d e5 jge a328 ibv_get_device_list@IBVERBS_1.0+0xa8 a363: 75 1f jne a384 ibv_get_device_list@IBVERBS_1.0+0x104 a373: 74 04 je a379 ibv_get_device_list@IBVERBS_1.0+0xf9 a37d: eb d6 jmp a355 ibv_get_device_list@IBVERBS_1.0+0xd5 a382: eb d1 jmp a355 ibv_get_device_list@IBVERBS_1.0+0xd5 000000000000ac00 ibv_get_device_list@@IBVERBS_1.1: ac14: 74 06 je ac1c ibv_get_device_list@@IBVERBS_1.1+0x1c ac30: 0f 84 aa 00 00 00 je ace0 ibv_get_device_list@@IBVERBS_1.1+0xe0 ac36: 0f 88 ca 00 00 00 js ad06 ibv_get_device_list@@IBVERBS_1.1+0x106 ac4d: 0f 88 c5 00 00 00 js ad18 ibv_get_device_list@@IBVERBS_1.1+0x118 ac69: 0f 84 b9 00 00 00 je ad28 ibv_get_device_list@@IBVERBS_1.1+0x128 ac87: 74 2a je acb3 ibv_get_device_list@@IBVERBS_1.1+0xb3 acb1: 75 dd jne ac90 ibv_get_device_list@@IBVERBS_1.1+0x90 acb6: 74 04 je acbc ibv_get_device_list@@IBVERBS_1.1+0xbc ace9: 78 15 js ad00 ibv_get_device_list@@IBVERBS_1.1+0x100 acf5: e9 42 ff ff ff jmpq ac3c ibv_get_device_list@@IBVERBS_1.1+0x3c ad12: eb a8 jmp acbc ibv_get_device_list@@IBVERBS_1.1+0xbc ad26: eb 94 jmp acbc ibv_get_device_list@@IBVERBS_1.1+0xbc ad33: eb 87 jmp acbc ibv_get_device_list@@IBVERBS_1.1+0xbc

venkatsc commented 5 years ago

configure: error: <infiniband/driver.h> not found. Is libibverbs installed?

Facing same issue. Did anyone found the solution for this issue?

carlboe commented 5 years ago

infiniband/driver.h is in libibverbs-dev package

For kernel 4.4.0-164 at least, the function get_user_pages() was back ported from a later kernel. You will find the necessary change in the commit history susequent to the 4.4 kernel release:

                        rv = get_user_pages(current, current->mm,
-                                           first_page_va, nents, 1, 1, plist,
+                                           first_page_va, nents, FOLL_WRITE,  plist,
                                            NULL);

I see elsewhere (https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/xenial/commit/?h=master-next&id=9c72ad08f1510332e9314046e6a2adba5064f39a) that (FOLL_WRITE | FOLL_FORCE ) seems to be the gup_flags replacement for the two arguments 1,1 , not just FOLL_WRITE , but I have no knowledge to make this judgement.

BernardMetzler commented 5 years ago

SoftiWarp is part of Linux since 5.3. So we finally made it into upstream. With that, I am going to cease my effort regarding this github repo. Sorry for not being really responsive during the last quarter. This due to the final heavy lifting getting it upstream. So please update your kernel to get the latest version. linux-rdma@vger.kernel.org will be the right place to discuss issues. See http://vger.kernel.org/vger-lists.html#linux-rdma for email list history and subscription process.

carlboe commented 5 years ago

Got it. I now have a 5.3 kernel sitting on top of a 18.04 installation of ubuntu. This has the needed siw module. I discovered that I still need an updated iproute2 and an updated rdma-core in order to link the siw to the ethernet device and for ibv_devices to list the device. I will try to write up the experience on the google group, as I still have questions.