namjaejeon
commented
1 year ago Have you tried load mlx and rdma core driver using modprobe ? sudo modprobe mlx5_core sudo modprobe mlx5_ib
Closed llamafilm closed 3 weeks ago
namjaejeon
commented
1 year ago Have you tried load mlx and rdma core driver using modprobe ? sudo modprobe mlx5_core sudo modprobe mlx5_ib
llamafilm
commented
1 year ago Yes, those are loaded already
# lsmod | grep mlx
mlx5_ib 446464 0
ib_uverbs 139264 2 rdma_ucm,mlx5_ib
ib_core 425984 8 rdma_cm,ib_ipoib,iw_cm,ib_umad,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm
mlx5_core 2015232 1 mlx5_ib
mlxdevm 176128 1 mlx5_core
mlxfw 32768 1 mlx5_core
psample 20480 1 mlx5_core
tls 114688 1 mlx5_core
mlx_compat 69632 11 rdma_cm,ib_ipoib,mlxdevm,iw_cm,ib_umad,ib_core,rdma_ucm,ib_uverbs,mlx5_ib,ib_cm,mlx5_coreBTW, are there any benefits to installing it as part of the kernel instead of as a module?
namjaejeon
commented
1 year ago these are ksmbd module dependancy. Please try to insmod these drivers. kernel/fs/ksmbd/ksmbd.ko: kernel/drivers/infiniband/core/rdma_cm.ko kernel/drivers/infiniband/core/iw_cm.ko kernel/drivers/infiniband/core/ib_cm.ko kernel/drivers/infiniband/core/ib_core.ko kernel/lib/crypto/libdes.ko
namjaejeon
commented
1 year ago BTW, are there any benefits to installing it as part of the kernel instead of as a module?
You don't need to insmod kernel module. but kernel image size is increased. So if you don't need to use this module, you can not unload it because it is built-in kernel. If HW has short of memory, Memory usage increases due to the large kernel image size including unused modules.
llamafilm
commented
1 year ago I checked, and those 5 modules are already loaded.
namjaejeon
commented
1 year ago Or you can command in ksmbd source.
sudo make install sudo modprobe ksmbd
[ 201.906576] ksmbd: module verification failed: signature and/or required key missing - tainting kernel
[ 201.906612] ksmbd: Unknown symbol ib_mr_pool_destroy (err -2)
[ 201.906642] ksmbd: Unknown symbol ib_unregister_client (err -2)
[ 201.906667] ksmbd: Unknown symbol rdma_rw_ctx_destroy (err -2)
[ 201.906674] ksmbd: Unknown symbol rdma_event_msg (err -2)
[ 201.906700] ksmbd: Unknown symbol __ib_alloc_pd (err -2)
[ 201.906740] ksmbd: Unknown symbol ib_mr_pool_init (err -2)
[ 201.906784] ksmbd: Unknown symbol ib_event_msg (err -2)
[ 201.906811] ksmbd: Unknown symbol rdma_disconnect (err -2)
[ 201.906858] ksmbd: Unknown symbol __rdma_create_kernel_id (err -2)
[ 201.906890] ksmbd: Unknown symbol ib_device_get_by_netdev (err -2)
[ 201.906898] ksmbd: Unknown symbol rdma_bind_addr (err -2)
[ 201.906905] ksmbd: Unknown symbol ib_register_client (err -2)
[ 201.906917] ksmbd: Unknown symbol rdma_create_qp (err -2)
[ 201.906930] ksmbd: Unknown symbol rdma_rw_ctx_wrs (err -2)
[ 201.906972] ksmbd: Unknown symbol __ib_alloc_cq (err -2)
[ 201.906990] ksmbd: Unknown symbol rdma_listen (err -2)
[ 201.907013] ksmbd: Unknown symbol ib_destroy_qp_user (err -2)
[ 201.907022] ksmbd: Unknown symbol des_encrypt (err -2)
[ 201.907050] ksmbd: Unknown symbol ib_destroy_cq_user (err -2)
[ 201.907067] ksmbd: Unknown symbol des_expand_key (err -2)
[ 201.907078] ksmbd: Unknown symbol ib_wc_status_msg (err -2)
[ 201.907091] ksmbd: Unknown symbol ib_dma_virt_map_sg (err -2)
[ 201.907107] ksmbd: Unknown symbol ib_free_cq (err -2)
[ 201.907116] ksmbd: Unknown symbol rdma_destroy_id (err -2)
[ 201.907134] ksmbd: Unknown symbol rdma_accept (err -2)
[ 201.907150] ksmbd: Unknown symbol ib_drain_qp (err -2)
[ 201.907156] ksmbd: Unknown symbol ib_dealloc_pd_user (err -2)
[ 201.907175] ksmbd: Unknown symbol ib_device_put (err -2)
[ 201.907200] ksmbd: Unknown symbol rdma_rw_ctx_init (err -2)
linkinjeon@linkinjeon-Precision-Tower-3620:~/git/smbd_work/ksmbd$ vi Makefile
linkinjeon@linkinjeon-Precision-Tower-3620:~/git/smbd_work/ksmbd$ sudo make install
rm -f /lib/modules/5.15.0-46-generic/kernel/fs/ksmbd/ksmbd.ko
install -m644 -b -D ksmbd.ko /lib/modules/5.15.0-46-generic/kernel/fs/ksmbd/ksmbd.ko
depmod -a
linkinjeon@linkinjeon-Precision-Tower-3620:~/git/smbd_work/ksmbd$ sudo modprobe ksmbd
linkinjeon@linkinjeon-Precision-Tower-3620:~/git/smbd_work/ksmbd$ lsmod | grep ksmbd
ksmbd 270336 0
rdma_cm 122880 1 ksmbd
ib_core 393216 4 rdma_cm,iw_cm,ksmbd,ib_cm
libdes 32768 1 ksmbdThat's essentially what I've been doing already. Which driver version do you use? Do you think there is some compatibility issue with Mellanox 5.7 driver version?
namjaejeon
commented
1 year ago No, I have ever installed Mellanox drivers before. ksmbd RDMA work fine to me on my ubuntu although Mellanox driver is not installed. Because there are mellanox drivers in linux kernel mainline. When I see your debug log, Errors are coming from rdma core driver, not mellanox driver. I can reproduce it when I just command "insmod ksmbd" without make install. and work with commanding "make install" and modprobe ksmbd to insmod rdma core drivers.
namjaejeon
commented
1 year ago Can you try to "make clean" and "make" again in ksmbd source ?
llamafilm
commented
1 year ago I tried make clean; git pull; make; make install, that didn't change the outcome.
I tried uninstalling the Mellanox driver but then the NICs disappear completely. Maybe I'll try a slightly older driver version.
namjaejeon
commented
1 year ago @llamafilm Okay:)
hclee
commented
1 year ago It seems the mellanox driver package installed rdma_cm, ib_core, ... Can you run modinfo ib_core and share the result?
namjaejeon
commented
1 year ago Mellanox 5.7 mean kernel version(i.e. linux 5.7 kernel) ? Maybe, there is linux 5.7 kernel source in mellanox package ?
hclee
commented
1 year ago Mellanox 5.7 mean kernel version(i.e. linux 5.7 kernel) ? Maybe, there is linux 5.7 kernel source in mellanox package ?
I don't know the meaning of the version, but there is all code of drivers/infiniband subdirectories in the package. And it seems that messages below means the symbol exists but the CRC checksums are different.
[58086.240531] ksmbd: disagrees about version of symbol ib_mr_pool_destroy [58086.240535] ksmbd: Unknown symbol ib_mr_pool_destroy (err -22)
hclee
commented
1 year ago Maybe have to compile ksmbd against rdma modules provided by the mellanox driver.
namjaejeon
commented
1 year ago In case of Chelsio driver package, The package contains the kernel source corresponding to the driver. So users need to build and install the kernel in package after installing the packages.
llamafilm
commented
1 year ago I don't think 5.7 refers to any kernel, it's just Mellanox own versioning number system. https://network.nvidia.com/products/ethernet-drivers/linux/mlnx_en/
@hclee you're right, the driver does include ib_core and some others. Here's a comparison of ib_core with inbox driver vs the latest. I'm not sure how to compile ksmbd against these other rdma modules.
# modinfo ib_core
filename: /lib/modules/5.15.0-46-generic/kernel/drivers/infiniband/core/ib_core.ko
alias: rdma-netlink-subsys-4
license: Dual BSD/GPL
description: core kernel InfiniBand API
author: Roland Dreier
alias: net-pf-16-proto-20
alias: rdma-netlink-subsys-5
srcversion: 90657E3789C1E7B0413615C
depends:
retpoline: Y
intree: Y
name: ib_core
vermagic: 5.15.0-46-generic SMP mod_unload modversions
sig_id: PKCS#7
signer: Build time autogenerated kernel key
sig_key: 7D:17:41:79:9B:FD:BB:C4:C9:58:54:28:ED:61:EB:C2:E4:6C:2E:06
sig_hashalgo: sha512
signature: 52:D1:3F:C7:2C:46:88:AB:98:D9:29:41:BB:44:EC:0F:75:53:95:48:
7A:F5:9A:1B:1D:E9:B8:7D:A7:0C:77:A5:FC:6E:1B:AE:E6:30:B4:0A:
DB:A6:A7:06:E2:D2:EC:BA:AC:B3:32:60:90:30:A8:59:91:0E:CB:8E:
76:30:75:A5:19:FA:EF:AE:88:65:85:E7:F2:0E:52:FB:2F:7C:B3:88:
E8:56:98:4D:5B:9F:81:FE:6D:FB:EF:47:A1:79:BF:16:5A:F8:5E:D4:
74:A5:54:B4:1F:70:8E:AE:E3:DE:D2:7F:90:74:33:FC:4D:CA:19:8C:
BD:7F:43:CD:C4:91:0B:0F:4A:E2:40:46:0C:49:C8:BE:EC:BC:AE:C6:
92:54:35:52:17:46:B8:E8:A2:31:6B:3E:44:9B:FB:BA:DD:9C:46:38:
8A:D7:EB:48:1B:C5:47:5E:F6:B1:DD:8C:79:01:27:AC:90:43:CD:AF:
D3:0B:DA:F9:E3:37:90:6B:A7:D4:46:70:6B:3D:C7:D4:6A:2F:E0:CC:
FA:B4:B7:B0:D5:F7:F9:7E:B2:92:71:99:C6:36:3D:60:39:A3:07:C9:
1B:04:86:CD:2F:0C:55:38:93:94:C0:EF:FA:DD:51:B3:33:D8:2F:2B:
E7:16:92:7E:DA:AE:F1:E3:A1:98:CD:AB:3F:84:57:AD:D1:89:83:30:
42:EA:2F:4E:B4:FB:93:A7:EF:34:7E:3F:87:CF:5C:F0:66:BC:8B:66:
89:C4:97:9A:E8:F1:7B:BD:F6:30:AE:96:DB:F4:DD:5D:D9:FD:EE:77:
BA:F0:E0:CC:C8:26:B3:4A:F9:AC:E3:0C:5D:BE:86:2F:6E:35:FC:EE:
11:A0:4E:C9:67:73:C7:73:79:34:1B:F6:C8:5F:DE:A4:6A:23:DB:98:
BE:4C:A7:98:10:4D:E4:45:BE:9C:12:C8:0D:5B:94:F2:8F:2B:68:E7:
6A:01:3B:E3:9F:53:27:A4:CE:AA:53:E8:55:55:25:60:1D:0F:19:F7:
46:F0:70:30:AA:0F:AF:83:89:E2:95:3B:71:A1:14:8A:5A:EA:A6:98:
1D:00:9B:7C:9E:E2:F3:90:C7:CE:2E:27:9C:22:FB:3A:E0:9E:EA:83:
B4:5C:73:30:68:9C:7F:92:83:12:C5:95:FD:C2:3E:F7:12:E8:5D:FE:
82:C1:0A:BC:F6:7C:93:40:47:4C:4F:F2:9C:5F:4E:F5:26:E8:01:FE:
6E:55:F2:0F:C1:2F:7A:CB:07:C5:B4:AB:E6:A4:9F:4E:5E:3C:95:51:
80:44:CF:6F:24:2F:FC:31:9C:C8:38:3A:26:59:EC:FB:86:C9:90:63:
90:1E:AD:75:92:21:2E:7D:8E:59:20:0E
parm: send_queue_size:Size of send queue in number of work requests (int)
parm: recv_queue_size:Size of receive queue in number of work requests (int)
parm: netns_mode:Share device among net namespaces; default=1 (shared) (bool)
parm: force_mr:Force usage of MRs for RDMA READ/WRITE operations (bool)# modinfo ib_core
filename: /lib/modules/5.15.0-46-generic/updates/dkms/ib_core.ko
alias: rdma-netlink-subsys-4
license: Dual BSD/GPL
description: core kernel InfiniBand API
author: Roland Dreier
alias: net-pf-16-proto-20
alias: rdma-netlink-subsys-5
import_ns: DMA_BUF
srcversion: 162C74CC0553CF4D9E2C6F0
depends: mlx_compat
retpoline: Y
name: ib_core
vermagic: 5.15.0-46-generic SMP mod_unload modversions
parm: send_queue_size:Size of send queue in number of work requests (int)
parm: recv_queue_size:Size of receive queue in number of work requests (int)
parm: mad_smp_window:Maximun number of outgoing SMP requests (int)
parm: netns_mode:Share device among net namespaces; default=1 (shared) (bool)
parm: force_mr:Force usage of MRs for RDMA READ/WRITE operations (bool)@llamafilm I have never compiled the kernel module against others. But modules.txt may help you.
Use an extra Module.symvers file
When an external module is built, a Module.symvers file
is generated containing all exported symbols which are
not defined in the kernel. To get access to symbols
from bar.ko, copy the Module.symvers file from the
compilation of bar.ko to the directory where foo.ko is
built. During the module build, kbuild will read the
Module.symvers file in the directory of the external
module, and when the build is finished, a new
Module.symvers file is created containing the sum of
all symbols defined and not part of the kernel.
Use "make" variable KBUILD_EXTRA_SYMBOLS
If it is impractical to copy Module.symvers from
another module, you can assign a space separated list
of files to KBUILD_EXTRA_SYMBOLS in your build file.
These files will be loaded by modpost during the
initialization of its symbol tables.
consp
commented
1 year ago I never got it working with the Mellanox drivers. You miss at minimum several function references which are not (correctly) ported from mainline to the Mellanox drivers and until Mellanox or Nvidia fixes this/adds the missing functions it is almost impossible to get working. I also tried making function aliases to avoid these issues but every fix gave me two more errors and I gave up and used the inbox kernel drivers.
You might want to try sending a bug report to Mellanox since they claim kernel 5.15+ support now.
btbxbob
commented
1 month ago Still not working today. For a record: mlnx_ofed driver version: MLNX_OFED_LINUX-24.04-0.6.6.0-ubuntu24.04-x86_64
❯ sudo ./mlnxofedinstall --add-kernel-support
# cd to ksmbd source folder
❯ cp /usr/src/ofa_kernel/default/Module.symvers .
❯ make clean
make -C /lib/modules/6.8.0-35-generic/build M=/home/bx/src/ksmbd clean
make[1]: Entering directory '/usr/src/linux-headers-6.8.0-35-generic'
CLEAN /home/bx/src/ksmbd/Module.symvers
make[1]: Leaving directory '/usr/src/linux-headers-6.8.0-35-generic'
❯ make -j16
make -C /lib/modules/6.8.0-35-generic/build M=/home/bx/src/ksmbd modules
make[1]: Entering directory '/usr/src/linux-headers-6.8.0-35-generic'
warning: the compiler differs from the one used to build the kernel
The kernel was built by: x86_64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0
You are using: gcc-13 (Ubuntu 13.2.0-23ubuntu4) 13.2.0
CC [M] /home/bx/src/ksmbd/unicode.o
CC [M] /home/bx/src/ksmbd/auth.o
CC [M] /home/bx/src/ksmbd/vfs.o
CC [M] /home/bx/src/ksmbd/vfs_cache.o
CC [M] /home/bx/src/ksmbd/connection.o
CC [M] /home/bx/src/ksmbd/crypto_ctx.o
CC [M] /home/bx/src/ksmbd/server.o
CC [M] /home/bx/src/ksmbd/misc.o
CC [M] /home/bx/src/ksmbd/oplock.o
CC [M] /home/bx/src/ksmbd/ksmbd_work.o
CC [M] /home/bx/src/ksmbd/smbacl.o
CC [M] /home/bx/src/ksmbd/ndr.o
CC [M] /home/bx/src/ksmbd/mgmt/ksmbd_ida.o
CC [M] /home/bx/src/ksmbd/mgmt/user_config.o
CC [M] /home/bx/src/ksmbd/mgmt/share_config.o
CC [M] /home/bx/src/ksmbd/mgmt/tree_connect.o
CC [M] /home/bx/src/ksmbd/mgmt/user_session.o
CC [M] /home/bx/src/ksmbd/smb_common.o
CC [M] /home/bx/src/ksmbd/transport_tcp.o
CC [M] /home/bx/src/ksmbd/transport_ipc.o
CC [M] /home/bx/src/ksmbd/smb2pdu.o
CC [M] /home/bx/src/ksmbd/smb2ops.o
CC [M] /home/bx/src/ksmbd/smb2misc.o
ASN.1 /home/bx/src/ksmbd/ksmbd_spnego_negtokeninit.asn1.[ch]
ASN.1 /home/bx/src/ksmbd/ksmbd_spnego_negtokentarg.asn1.[ch]
CC [M] /home/bx/src/ksmbd/compat.o
CC [M] /home/bx/src/ksmbd/transport_rdma.o
CC [M] /home/bx/src/ksmbd/ksmbd_spnego_negtokeninit.asn1.o
CC [M] /home/bx/src/ksmbd/ksmbd_spnego_negtokentarg.asn1.o
CC [M] /home/bx/src/ksmbd/asn1.o
LD [M] /home/bx/src/ksmbd/ksmbd.o
MODPOST /home/bx/src/ksmbd/Module.symvers
CC [M] /home/bx/src/ksmbd/ksmbd.mod.o
LD [M] /home/bx/src/ksmbd/ksmbd.ko
BTF [M] /home/bx/src/ksmbd/ksmbd.ko
Skipping BTF generation for /home/bx/src/ksmbd/ksmbd.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-6.8.0-35-generic'
❯ sudo make install
rm -f /lib/modules/6.8.0-35-generic/kernel/fs/ksmbd/ksmbd.ko
install -m644 -b -D ksmbd.ko /lib/modules/6.8.0-35-generic/kernel/fs/ksmbd/ksmbd.ko
depmod -a
❯ sudo modprobe ksmbd
modprobe: ERROR: could not insert 'ksmbd': Invalid argument@btbxbob please share dmesg result.
namjaejeon
commented
1 month ago @btbxbob You can use mainline ksmbd instead of this github ksmbd. Is there any reason ?
btbxbob
commented
1 month ago @btbxbob You can use mainline ksmbd instead of this github ksmbd. Is there any reason ?
I am debugging the smb direct feature with another windows client. The kernel is reconfigured too.
I think I have tried the mainline kernel ksmbd with the same problem, not so sure.
namjaejeon
commented
1 month ago I think I have tried the mainline kernel ksmbd with the same problem, not so sure.
Can you share dmesg result when you do modprobe mainline ksmbd ?
btbxbob
commented
1 month ago start from clean mainline kernel, with default build config
❯ uname -r
6.9.3-060903-generic
❯ sudo modprobe ksmbdload ok with no output install mlnx_ofed driver
❯ sudo ./mlnxofedinstall --dkmsinstall success, then reload mods
❯ sudo /etc/init.d/openibd restart
Unloading rdma_cm [FAILED]
rmmod: ERROR: Module rdma_cm is in use by: ksmbd
❯ sudo modprobe -r ksmbd
❯ sudo /etc/init.d/openibd restart
Unloading HCA driver: [ OK ]
Loading HCA driver and Access Layer: [ OK ]
❯ sudo modprobe ksmbd
modprobe: ERROR: could not insert 'ksmbd': Invalid argumendmesg:
[ 2229.396165] mlx5_core 0000:01:00.0 enp1s0f0np0: Link up
[ 2229.593454] mlx5_core 0000:01:00.1 enp1s0f1np1: Link down
[ 2247.026770] ksmbd: disagrees about version of symbol ib_mr_pool_destroy
[ 2247.026774] ksmbd: Unknown symbol ib_mr_pool_destroy (err -22)
[ 2247.026784] ksmbd: disagrees about version of symbol ib_unregister_client
[ 2247.026785] ksmbd: Unknown symbol ib_unregister_client (err -22)
[ 2247.026796] ksmbd: disagrees about version of symbol rdma_rw_ctx_destroy
[ 2247.026797] ksmbd: Unknown symbol rdma_rw_ctx_destroy (err -22)
[ 2247.026808] ksmbd: disagrees about version of symbol __ib_alloc_pd
[ 2247.026808] ksmbd: Unknown symbol __ib_alloc_pd (err -22)
[ 2247.026825] ksmbd: disagrees about version of symbol ib_mr_pool_init
[ 2247.026826] ksmbd: Unknown symbol ib_mr_pool_init (err -22)
[ 2247.026858] ksmbd: disagrees about version of symbol ib_event_msg
[ 2247.026860] ksmbd: Unknown symbol ib_event_msg (err -22)
[ 2247.026867] ksmbd: disagrees about version of symbol rdma_disconnect
[ 2247.026867] ksmbd: Unknown symbol rdma_disconnect (err -22)
[ 2247.026870] ksmbd: disagrees about version of symbol ib_find_gid
[ 2247.026871] ksmbd: Unknown symbol ib_find_gid (err -22)
[ 2247.026891] ksmbd: disagrees about version of symbol __rdma_create_kernel_id
[ 2247.026893] ksmbd: Unknown symbol __rdma_create_kernel_id (err -22)
[ 2247.026909] ksmbd: disagrees about version of symbol ib_device_get_by_netdev
[ 2247.026910] ksmbd: Unknown symbol ib_device_get_by_netdev (err -22)
[ 2247.026912] ksmbd: disagrees about version of symbol rdma_bind_addr
[ 2247.026913] ksmbd: Unknown symbol rdma_bind_addr (err -22)
[ 2247.026915] ksmbd: disagrees about version of symbol ib_register_client
[ 2247.026916] ksmbd: Unknown symbol ib_register_client (err -22)
[ 2247.026922] ksmbd: disagrees about version of symbol rdma_create_qp
[ 2247.026923] ksmbd: Unknown symbol rdma_create_qp (err -22)
[ 2247.026930] ksmbd: disagrees about version of symbol rdma_rw_ctx_wrs
[ 2247.026931] ksmbd: Unknown symbol rdma_rw_ctx_wrs (err -22)
[ 2247.026955] ksmbd: disagrees about version of symbol __ib_alloc_cq
[ 2247.026956] ksmbd: Unknown symbol __ib_alloc_cq (err -22)
[ 2247.026966] ksmbd: disagrees about version of symbol rdma_listen
[ 2247.026967] ksmbd: Unknown symbol rdma_listen (err -22)
[ 2247.026986] ksmbd: disagrees about version of symbol ib_destroy_qp_user
[ 2247.026987] ksmbd: Unknown symbol ib_destroy_qp_user (err -22)
[ 2247.027008] ksmbd: disagrees about version of symbol ib_destroy_cq_user
[ 2247.027008] ksmbd: Unknown symbol ib_destroy_cq_user (err -22)
[ 2247.027023] ksmbd: disagrees about version of symbol ib_wc_status_msg
[ 2247.027023] ksmbd: Unknown symbol ib_wc_status_msg (err -22)
[ 2247.027034] ksmbd: disagrees about version of symbol ib_dma_virt_map_sg
[ 2247.027035] ksmbd: Unknown symbol ib_dma_virt_map_sg (err -22)
[ 2247.027046] ksmbd: disagrees about version of symbol ib_free_cq
[ 2247.027047] ksmbd: Unknown symbol ib_free_cq (err -22)
[ 2247.027051] ksmbd: disagrees about version of symbol rdma_destroy_id
[ 2247.027052] ksmbd: Unknown symbol rdma_destroy_id (err -22)
[ 2247.027068] ksmbd: disagrees about version of symbol rdma_accept
[ 2247.027068] ksmbd: Unknown symbol rdma_accept (err -22)
[ 2247.027083] ksmbd: disagrees about version of symbol ib_drain_qp
[ 2247.027084] ksmbd: Unknown symbol ib_drain_qp (err -22)
[ 2247.027087] ksmbd: disagrees about version of symbol ib_dealloc_pd_user
[ 2247.027088] ksmbd: Unknown symbol ib_dealloc_pd_user (err -22)
[ 2247.027112] ksmbd: disagrees about version of symbol ib_device_put
[ 2247.027113] ksmbd: Unknown symbol ib_device_put (err -22)
[ 2247.027156] ksmbd: disagrees about version of symbol rdma_rw_ctx_init
[ 2247.027157] ksmbd: Unknown symbol rdma_rw_ctx_init (err -22)Can you tell me why you need to check smb-direct in ksmbd first ?
namjaejeon
commented
1 month ago This is not ksmbd issue. So I can't tell you what is problem. please google with "disagrees about version of symbol" keyword. maybe. Please disable "module versioning support" in menuconfig if you load other version rdma modules.
btbxbob
commented
1 month ago Can you tell me why you need to check smb-direct in ksmbd first ?
well, mostly because i am using windows client? i'm trying to speed up my nas speed, it's a hobby project.
This is not ksmbd issue. So I can't tell you what is problem. please google with "disagrees about version of symbol" keyword. maybe. Please disable "module versioning support" in menuconfig if you load other version rdma modules.
will do.
btbxbob
commented
1 month ago This is not ksmbd issue. So I can't tell you what is problem. please google with "disagrees about version of symbol" keyword. maybe. Please disable "module versioning support" in menuconfig if you load other version rdma modules.
This resolve my problem. I think it also resolve the origin one.
Though i can't enable rdma with the mlnx_ofed driver. I can enable with the inbox driver.
I had this partially working before with the Mellanox inbox driver. Then I upgraded to the latest MLNX-EN driver 5.7-1.0.2.0 to see if that would help. By default, the Mellanox
install.shscript does not enable RDMA, which I discovered becauserpingfails:So I re-ran the installer with
--vmaoption. This way,rpingworks with size up to 1444 bytes. But I'm unable to load the ksmbd module:dmesg:
I'm using the stock kernel in Ubuntu 22.04 and the kernel version matches the module version exactly, so I don't understand why it the symbol versions don't match.
I don't have any other kernel versions. I think I have all the right headers and sources: