Fedora 33 mmap failing

[adivyoseph]

on Fedora 33 lspci b3:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Subsystem: Mellanox Technologies ConnectX-4 Lx Stand-up dual-port 10GbE MCX4121A-XCAT Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 37 NUMA node: 0 Region 0: Memory at 3800fe000000 (64-bit, prefetchable) [size=32M] Expansion ROM at fbe00000 [disabled] [size=1M] Capabilities: Kernel driver in use: mlx5_core Kernel modules: mlx5_core

ls /sys/bus/pci/devices/0000:b3:00.0/ -l total 0 -r--r--r--. 1 root root 4096 Mar 1 11:41 aer_dev_correctable -r--r--r--. 1 root root 4096 Mar 1 11:41 aer_dev_fatal -r--r--r--. 1 root root 4096 Mar 1 11:41 aer_dev_nonfatal -r--r--r--. 1 root root 4096 Feb 21 09:25 ari_enabled -rw-r--r--. 1 root root 4096 Mar 1 11:41 broken_parity_status -r--r--r--. 1 root root 4096 Feb 21 09:25 class -rw-r--r--. 1 root root 4096 Feb 21 09:25 config -r--r--r--. 1 root root 4096 Mar 1 11:41 consistent_dma_mask_bits -r--r--r--. 1 root root 4096 Mar 1 11:41 current_link_speed -r--r--r--. 1 root root 4096 Mar 1 11:41 current_link_width -rw-r--r--. 1 root root 4096 Mar 1 11:41 d3cold_allowed -r--r--r--. 1 root root 4096 Feb 21 09:25 device -r--r--r--. 1 root root 4096 Mar 1 11:41 dma_mask_bits lrwxrwxrwx. 1 root root 0 Feb 21 09:25 driver -> ../../../../bus/pci/drivers/mlx5_core -rw-r--r--. 1 root root 4096 Mar 1 11:41 driver_override -rw-r--r--. 1 root root 4096 Mar 1 11:41 enable lrwxrwxrwx. 1 root root 0 Mar 1 11:41 firmware_node -> ../../../LNXSYSTM:00/LNXSYBUS:00/PNP0A08:03/device:b6/device:b9 drwxr-xr-x. 16 root root 0 Feb 21 09:25 infiniband drwxr-xr-x. 4 root root 0 Feb 21 09:25 infiniband_mad drwxr-xr-x. 3 root root 0 Feb 21 09:25 infiniband_verbs -r--r--r--. 1 root root 4096 Mar 1 10:13 irq drwxr-xr-x. 2 root root 0 Feb 24 14:45 link -r--r--r--. 1 root root 4096 Mar 1 11:41 local_cpulist -r--r--r--. 1 root root 4096 Mar 1 11:41 local_cpus -r--r--r--. 1 root root 4096 Mar 1 11:41 max_link_speed -r--r--r--. 1 root root 4096 Mar 1 11:41 max_link_width -r--r--r--. 1 root root 4096 Mar 1 10:13 modalias -rw-r--r--. 1 root root 4096 Mar 1 11:41 msi_bus drwxr-xr-x. 2 root root 0 Feb 24 14:45 msi_irqs drwxr-xr-x. 3 root root 0 Feb 21 09:25 net -rw-r--r--. 1 root root 4096 Feb 21 09:25 numa_node -r--r--r--. 1 root root 4096 Mar 1 11:41 pools drwxr-xr-x. 2 root root 0 Feb 24 14:45 power drwxr-xr-x. 3 root root 0 Feb 21 09:25 ptp --w--w----. 1 root root 4096 Mar 1 11:41 remove --w-------. 1 root root 4096 Mar 1 11:41 rescan --w-------. 1 root root 4096 Mar 1 11:41 reset -r--r--r--. 1 root root 4096 Feb 21 09:25 resource -rw-------. 1 root root 33554432 Mar 1 11:41 resource0 -rw-------. 1 root root 33554432 Mar 1 11:41 resource0_wc -r--r--r--. 1 root root 4096 Feb 21 09:25 revision -rw-------. 1 root root 1048576 Feb 21 09:25 rom -rw-r--r--. 1 root root 4096 Mar 1 11:41 sriov_drivers_autoprobe -rw-r--r--. 1 root root 4096 Feb 21 09:25 sriov_numvfs -r--r--r--. 1 root root 4096 Mar 1 11:41 sriov_offset -r--r--r--. 1 root root 4096 Mar 1 11:41 sriov_stride -r--r--r--. 1 root root 4096 Feb 21 09:25 sriov_totalvfs -r--r--r--. 1 root root 4096 Mar 1 11:41 sriov_vf_device lrwxrwxrwx. 1 root root 0 Feb 21 09:25 subsystem -> ../../../../bus/pci -r--r--r--. 1 root root 4096 Feb 21 09:25 subsystem_device -r--r--r--. 1 root root 4096 Feb 21 09:25 subsystem_vendor -rw-r--r--. 1 root root 4096 Feb 21 09:25 uevent -r--r--r--. 1 root root 4096 Feb 21 09:25 vendor -rw-------. 1 root root 0 Mar 1 11:41 vpd

program output sudo ./pcimem /sys/bus/pci/devices/0000:b3:00.0/resource0_wc 0 w /sys/bus/pci/devices/0000:b3:00.0/resource0_wc opened. Target offset is 0x0, page size is 4096 mmap(0, 4096, 0x3, 0x1, 3, 0x0) Error at line 111, file pcimem.c (22) [Invalid argument]

Any ideas?

[adivyoseph]

found the problem, not invalid arguments. Had to unbind and existing device driver first, then mmap worked

[marc-hb]

Looks like a kernel config but which one? It goes away when running a custom kernel (with totally different hardware)

[marc-hb]

CONFIG_IO_STRICT_DEVMEM or CONFIG_STRICT_DEVMEM