search

RemixVSL / iomemory-vsl4

Updated Fusion-io iomemory VSL4 Linux (version 4.3.7) driver for recent kernels.
55 stars 9 forks source link

Build failed with Linux 5.15 and 5.16 on Slackware64 15.0 #39

Closed w41l closed 2 years ago

w41l commented 2 years ago

Bug description

Build failed with Linux 5.15 and 5.16 on Slackware64 15.0. Build normally with Linux 5.13.19.

How to reproduce

git clone https://github.com/snuf/iomemory-vsl4.git
cd iomemory-vsl4
cp tools/udev/rules.d/60-persistent-fio.rules /lib/udev/rules.d/
cp root/etc/ld.so.conf.d/fio.conf /etc/ld.so.conf.d/
cp -r root/usr/lib/* /usr/lib/
make module
**
/usr/src/iomemory-vsl4-ga03d9e0/root/usr/src/iomemory-vsl4-4.3.7/include/kblock_meta.h:58:20: error: implicit declaration of function ‘bdgrab’; did you mean ‘igrab’? [-Werror=implicit-function-declaration]
   58 |   #define GET_BDEV bdgrab(disk->gd->part0);
      |                    ^~~~~~
/usr/src/iomemory-vsl4-ga03d9e0/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:814:26: note: in expansion of macro ‘GET_BDEV’
  814 |             linux_bdev = GET_BDEV;
      |                          ^~~~~~~~
/usr/src/iomemory-vsl4-ga03d9e0/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:814:24: warning: assignment to ‘struct block_device *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  814 |             linux_bdev = GET_BDEV;
      |                        ^
/usr/src/iomemory-vsl4-ga03d9e0/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:894:40: error: ‘struct block_device’ has no member named ‘bd_mutex’
  894 |                  mutex_lock(&linux_bdev->bd_mutex);
      |                                        ^~
/usr/src/iomemory-vsl4-ga03d9e0/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:895:42: error: ‘struct block_device’ has no member named ‘bd_mutex’
  895 |                  mutex_unlock(&linux_bdev->bd_mutex);
      |                                          ^~
**

Possible solution

I believe bdgrab was dropped in Linux 5.15. For example in this commit message

Environment information

Information about the system the module is used on

  1. Linux: 5.15.27 and 5.16.17
  2. Compiler: GCC 11.2.0
  3. OS: Slackware64 15.0
  4. Source branch: main
  5. Device (using 5.13.19): 
    
root@hv32:~# fio-status

Found 1 VSL driver package: 4.3.7 build 1205 Driver: loaded

Found 2 ioMemory devices in this system

Adapter: ioMono (driver 4.3.7) IBM 3200GB Enterprise Value io3 Flash Adapter for System x, Product Number:00D8432, SN:11S00D8436Y050EB4C9039 PCIe Power limit threshold: 74.75W Connected ioMemory modules: fct0: 11:00.0, Product Number:00D8432, SN:11S00D8436Y050EB4C9039

fct0 Attached ioMemory Adapter Controller, Product Number:00D8432, SN:1447G0149 PCI:11:00.0, Slot Number:9 Firmware v8.9.8, rev 20161119 Public 3200.00 GBytes device size Internal temperature: 37.90 degC, max 43.80 degC Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00% Contained Virtual Partitions: fioa: ID:0, UUID:a068be9b-9ba8-4937-b544-850259b1bc30

fioa State: Online, Type: block device, Device: /dev/fioa ID:0, UUID:a068be9b-9ba8-4937-b544-850259b1bc30 3200.00 GBytes device size

Adapter: ioMono (driver 4.3.7) IBM 3200GB Enterprise Value io3 Flash Adapter for System x, Product Number:00D8432, SN:11S00D8436Y050EB4C9046 PCIe Power limit threshold: 74.75W Connected ioMemory modules: fct1: 41:00.0, Product Number:00D8432, SN:11S00D8436Y050EB4C9046

fct1 Attached ioMemory Adapter Controller, Product Number:00D8432, SN:1447G0172 PCI:41:00.0, Slot Number:8 Firmware v8.9.8, rev 20161119 Public 3200.00 GBytes device size Internal temperature: 46.76 degC, max 53.65 degC Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00% Contained Virtual Partitions: fiob: ID:0, UUID:5dc81890-7ade-4db5-8e91-3566e1714fc2

fiob State: Online, Type: block device, Device: /dev/fiob ID:0, UUID:5dc81890-7ade-4db5-8e91-3566e1714fc2 3200.00 GBytes device size

w41l commented 2 years ago

Additional info on bdgrab removal https://marc.info/?l=linux-block&m=163033396024336

snuf commented 2 years ago

@w41l thanks for the bug report. Yes it was, and there is a check that should find that: here. Can you post the output of the tests, the full output of the make module? Can you also post which tag are you on?

w41l commented 2 years ago

Thanks for the response. This is the log after re-cloned and rebuild


# git describe --tags --all --long
heads/main-0-g265a128

# make module
cd root/usr/src/iomemory-vsl4-4.3.7 && \
    make clean
make[1]: Entering directory '/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7'
make \
        -j144 \
    -C /lib/modules/5.16.17-wls/build \
    FIO_DRIVER_NAME=iomemory-vsl4 \
    FUSION_DRIVER_DIR=/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7 \
    M=/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7 \
    EXTRA_CFLAGS+="-I/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/include -DBUILDING_MODULE -DLINUX_IO_SCHED -Wall" \
    KFIO_LIB=kfio/x86_64_cc112_libkfio.o_shipped \
    clean
make[2]: Entering directory '/usr/src/linux-5.16.17'
make[2]: Leaving directory '/usr/src/linux-5.16.17'
rm -rf include/fio/port/linux/kfio_config.h kfio_config license.c
make[1]: Leaving directory '/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7'
cd root/usr/src/iomemory-vsl4-4.3.7 && \
    make gpl
make[1]: Entering directory '/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7'
sed -i 's/Proprietary/GPL/g' Kbuild

Change found in target kernel: KERNELVER KERNEL_SRC 
Running clean before building driver

make[2]: Entering directory '/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7'
make \
        -j144 \
    -C /lib/modules/5.16.17-wls/build \
    FIO_DRIVER_NAME=iomemory-vsl4 \
    FUSION_DRIVER_DIR=/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7 \
    M=/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7 \
    EXTRA_CFLAGS+="-I/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/include -DBUILDING_MODULE -DLINUX_IO_SCHED -Wall" \
    KFIO_LIB=kfio/x86_64_cc112_libkfio.o_shipped \
    clean
make[3]: Entering directory '/usr/src/linux-5.16.17'
make[3]: Leaving directory '/usr/src/linux-5.16.17'
rm -rf include/fio/port/linux/kfio_config.h kfio_config license.c
make[2]: Leaving directory '/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7'
if [ "112" -gt "75" ];then \
    if [ ! -f "kfio/x86_64_cc112_libkfio.o_shipped" ];then \
        cp kfio/x86_64_cc75_libkfio.o_shipped kfio/x86_64_cc112_libkfio.o_shipped; \
    fi \
fi
./kfio_config.sh -a x86_64 -o include/fio/port/linux/kfio_config.h -k /lib/modules/5.16.17-wls/build -p -d /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfio_config -l 0 -s /lib/modules/5.16.17-wls/source
Detecting Kernel Flags
Config dir         : /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfio_config
Output file        : include/fio/port/linux/kfio_config.h
Kernel output dir  : /lib/modules/5.16.17-wls/build
Kernel source dir  : /lib/modules/5.16.17-wls/source
Starting tests:
  1648425689.691  KFIOC_X_PROC_CREATE_DATA_WANTS_PROC_OPS...
  1648425689.723  KFIOC_X_TASK_HAS_CPUS_MASK...
  1648425689.756  KFIOC_X_LINUX_HAS_PART_STAT_H...
  1648425689.790  KFIOC_X_BLK_ALLOC_QUEUE_NODE_EXISTS...
  1648425689.825  KFIOC_X_BLK_ALLOC_DISK_EXISTS...
  1648425689.861  KFIOC_X_HAS_MAKE_REQUEST_FN...
  1648425689.897  KFIOC_X_GENHD_PART0_IS_A_POINTER...
  1648425689.933  KFIOC_X_BIO_HAS_BI_BDEV...
  1648425689.970  KFIOC_X_SUBMIT_BIO_RETURNS_BLK_QC_T...
  1648425690.004  KFIOC_X_VOID_ADD_DISK...
Started tests, waiting for completions...
  1648425691.078  KFIOC_X_PROC_CREATE_DATA_WANTS_PROC_OPS=1
  1648425691.113  KFIOC_X_TASK_HAS_CPUS_MASK=1
  1648425691.146  KFIOC_X_LINUX_HAS_PART_STAT_H=1
  1648425691.180  KFIOC_X_BLK_ALLOC_QUEUE_NODE_EXISTS=0
  1648425691.214  KFIOC_X_BLK_ALLOC_DISK_EXISTS=1
  1648425691.248  KFIOC_X_HAS_MAKE_REQUEST_FN=0
  1648425691.281  KFIOC_X_GENHD_PART0_IS_A_POINTER=1
  1648425691.314  KFIOC_X_BIO_HAS_BI_BDEV=1
  1648425691.348  KFIOC_X_SUBMIT_BIO_RETURNS_BLK_QC_T=0
  1648425691.382  KFIOC_X_VOID_ADD_DISK=0
Finished
1648425691.399  Exiting
Preserving configdir due to '-p' option: /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfio_config
grep: license.c: No such file or directory

Adding module version 5.16.17-265a128-4.3.7 to source license.c

make \
    -j144 \
-C /lib/modules/5.16.17-wls/build \
FIO_DRIVER_NAME=iomemory-vsl4 \
FUSION_DRIVER_DIR=/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7 \
M=/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7 \
EXTRA_CFLAGS+="-I/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/include -DBUILDING_MODULE -DLINUX_IO_SCHED -Wall" \
INSTALL_MOD_DIR=extra/fio \
INSTALL_MOD_PATH= \
KFIO_LIB=kfio/x86_64_cc112_libkfio.o_shipped \
modules
make[2]: Entering directory '/usr/src/linux-5.16.17'
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/main.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/dbgset.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/license.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/pci.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/sysrq.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/driver_init.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfio.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/errno.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/state.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kmsg.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/six_lock.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcache.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfile.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kmem.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kscatter.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/ktime.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/sched.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/cdev.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kscsi.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kscsi_host.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcpu.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcondvar.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/common_kinfo.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kinfo.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kinit.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfio_common.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/khotplug.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcsr.o
  SHIPPED /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kfio/x86_64_cc112_libkfio.o
  CC [M]  /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/module_param.o
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcpu.c: In function ‘kfio_map_cpus_to_read_queues’:
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcpu.c:214:5: warning: ISO C90 forbids variable length array ‘node_hist’ [-Wvla]
  214 |     uint32_t node_hist[nodes_possible], node_map[nodes_possible];
      |     ^~~~~~~~
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kcpu.c:214:5: warning: ISO C90 forbids variable length array ‘node_map’ [-Wvla]
In file included from /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:56:
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.c: In function ‘linux_bdev_hide_disk’:
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/include/kblock_meta.h:57:20: error: implicit declaration of function ‘bdgrab’; did you mean ‘igrab’? [-Werror=implicit-function-declaration]
   57 |   #define GET_BDEV bdgrab(disk->gd->part0);
      |                    ^~~~~~
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:813:26: note: in expansion of macro ‘GET_BDEV’
  813 |             linux_bdev = GET_BDEV;
      |                          ^~~~~~~~
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:813:24: warning: assignment to ‘struct block_device *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  813 |             linux_bdev = GET_BDEV;
      |                        ^
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:893:40: error: ‘struct block_device’ has no member named ‘bd_mutex’
  893 |                  mutex_lock(&linux_bdev->bd_mutex);
      |                                        ^~
/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.c:894:42: error: ‘struct block_device’ has no member named ‘bd_mutex’
  894 |                  mutex_unlock(&linux_bdev->bd_mutex);
      |                                          ^~
cc1: some warnings being treated as errors
make[3]: *** [scripts/Makefile.build:287: /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7/kblock.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [Makefile:1846: /usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7] Error 2
make[2]: Leaving directory '/usr/src/linux-5.16.17'
make[1]: *** [Makefile:136: modules] Error 2
make[1]: Leaving directory '/usr/src/iomemory-vsl4/root/usr/src/iomemory-vsl4-4.3.7'
make: *** [Makefile:29: module] Error 2
snuf commented 2 years ago

@w41l thanks for this! I seem to have swapped the macro for the GET_BDEV. There is also the matter of bd_mutex being removed and having moved to disk->open_mutex. I've created a branch bug-39 which contains these fixes. It compiles but I can't test it atm as my main machine is not running a 5.15/5.16 kernel. The open_mutex is important for detachment and unloading of the module.

w41l commented 2 years ago

I will try the branch bug-39 as soon as I can reboot the server

w41l commented 2 years ago

It crashed on linux_bdev_update_stats. Since I can't dump the boot log, I've captured the boot process as a video

snuf commented 2 years ago

Ah it's the queue creation that should be fixed. I remember to have tested the fix, so am kind of stumped. Progress is slow as there is no "fitting" test card atm (missing bracket).

snuf commented 2 years ago

Finally had some time to look and swapped the cards. pushed a fix to bug-39 which works on 5.15, and is going through testing on 5.15 now, will run test on 5.16 after that.

Apr 13 23:52:30 fio2 kernel: [  334.467804] fioinf ioDrive driver 5.15.13-77cf6ec-4.3.7.1205    loading...
Apr 13 23:52:30 fio2 kernel: [  334.468443] fioinf VSL configuration hash: f7771666cf9469344233dd611c787828ea46645e
Apr 13 23:52:30 fio2 kernel: [  334.495150] fioinf ioDrive 0000:00:04.0: mapping controller on BAR 5
Apr 13 23:52:30 fio2 kernel: [  334.644320] fioinf ioDrive 0000:00:04.0: Configuring controller.
Apr 13 23:52:30 fio2 kernel: [  334.662613] fioinf ioDrive 0000:00:04.0: using MSI-X interrupts
Apr 13 23:52:30 fio2 kernel: [  334.663083] fioinf ioDrive 0000:00:04.0: Starting controller
Apr 13 23:52:31 fio2 kernel: [  335.207356] fioinf ioDrive 0000:00:04.0: PMP Address: 1 1 1
Apr 13 23:52:31 fio2 kernel: [  335.787640] fioinf ioDrive 0000:00:04.0: SMP       fpga Microcode BOOT version 0.8.0 1
Apr 13 23:52:31 fio2 kernel: [  335.788041] fioinf ioDrive 0000:00:04.0: SMP controller Microcode APP  version 0.44.0 0
Apr 13 23:52:31 fio2 kernel: [  336.127686] fioinf ioDrive 0000:00:04.0: Product UUID is d3e0a40c-7e14-5264-842f-316d3f353492
Apr 13 23:52:31 fio2 kernel: [  336.128481] fioinf ioDrive 0000:00:04.0: Required PCIE bandwidth 4.000 GBytes per sec
Apr 13 23:52:31 fio2 kernel: [  336.129199] fioinf ioDrive 0000:00:04.0: Board serial number is 1446G1531
Apr 13 23:52:31 fio2 kernel: [  336.129822] fioinf ioDrive 0000:00:04.0: Adapter serial number is 1446G1531
Apr 13 23:52:31 fio2 kernel: [  336.130469] fioinf ioDrive 0000:00:04.0: Default capacity        1250.000 GBytes
Apr 13 23:52:31 fio2 kernel: [  336.131142] fioinf ioDrive 0000:00:04.0: Default sector size     4096 bytes
Apr 13 23:52:31 fio2 kernel: [  336.131791] fioinf ioDrive 0000:00:04.0: Rated endurance         4.00 PBytes
Apr 13 23:52:31 fio2 kernel: [  336.132448] fioinf ioDrive 0000:00:04.0: 85C temp range hardware found
Apr 13 23:52:31 fio2 kernel: [  336.133046] fioinf ioDrive 0000:00:04.0: Maximum capacity        1250.000 GBytes
Apr 13 23:52:31 fio2 kernel: [  336.133823] fioinf ioDrive 0000:00:04.0: PCIe Slot reported power limit: 0mWatts
Apr 13 23:52:31 fio2 kernel: [  336.134500] fioinf ioDrive 0000:00:04.0: Warning! PCIe Slot reported power 0mWatts is less than product minimum power 13000mWatts
Apr 13 23:52:31 fio2 kernel: [  336.135566] fioinf ioDrive 0000:00:04.0: PCIe Adapter power limit: 25000mWatts
Apr 13 23:52:31 fio2 kernel: [  336.136233] fioinf ioDrive 0000:00:04.0: PCIe Adapter power Throttle point: 24750mWatts
Apr 13 23:52:31 fio2 kernel: [  336.136985] fioinf ioDrive 0000:00:04.0: Firmware Archive Information:
Apr 13 23:52:31 fio2 kernel: [  336.137579] fioinf ioDrive 0000:00:04.0:   * Name        = ioMemory Firmware 8.9.9.20191010
Apr 13 23:52:31 fio2 kernel: [  336.138343] fioinf ioDrive 0000:00:04.0:   * Description = ioMemory Firmware 8.9.9.20191010 Archive
Apr 13 23:52:31 fio2 kernel: [  336.139174] fioinf ioDrive 0000:00:04.0:   * Version     = 8.9.9.0
Apr 13 23:52:31 fio2 kernel: [  336.139754] fioinf ioDrive 0000:00:04.0:   * Date        = 10/10/2019  Rev 20191010
Apr 13 23:52:32 fio2 kernel: [  336.140458] fioinf ioDrive 0000:00:04.0: Firmware version 8.9.9 118194 (0x802409 0x1cdb2)
Apr 13 23:52:32 fio2 kernel: [  336.141198] fioinf ioDrive 0000:00:04.0: Platform version 41 
Apr 13 23:52:32 fio2 kernel: [  336.141720] fioinf ioDrive 0000:00:04.0: Firmware VCS version 118194 [0x1cdb2]
Apr 13 23:52:32 fio2 kernel: [  336.142387] fioinf ioDrive 0000:00:04.0: Firmware VCS uid 0xea8892fee031ff5085ff47b5d3b4717e7dd4b603
Apr 13 23:52:32 fio2 kernel: [  336.161237] fioinf ioDrive 0000:00:04.0: Powercut flush: Supported and Enabled.
Apr 13 23:52:32 fio2 kernel: [  336.678042] fioinf ioDrive 0000:00:04.0: Loading microcode image 1.0.9 rev 100764 with flags 0x00010100
Apr 13 23:52:32 fio2 kernel: [  336.678566] fioinf ioDrive 0000:00:04.0: Microcode loaded (0).
Apr 13 23:52:32 fio2 kernel: [  336.720875] fioinf ioDrive 0000:00:04.0: Limiting to 4 banks
Apr 13 23:52:32 fio2 kernel: [  336.721134] fioinf ioDrive 0000:00:04.0: Multiple queues enabled
Apr 13 23:52:32 fio2 kernel: [  336.735554] fioinf ioDrive 0000:00:04.0: PCIe power monitor enabled (master). Limit set to 24.750 watts.
Apr 13 23:52:32 fio2 kernel: [  336.735993] fioinf ioDrive 0000:00:04.0: Thermal monitoring: Enabled
Apr 13 23:52:32 fio2 kernel: [  336.736274] fioinf ioDrive 0000:00:04.0: Hardware temperature alarm set for 85C.
Apr 13 23:52:32 fio2 kernel: [  336.791267] fioinf ioDrive 0000:00:04.0: Starting device ioMemory SX300-1300 0000:00:04.0
Apr 13 23:52:33 fio2 kernel: [  337.687020] fioinf ioMemory SX300-1300 0000:00:04.0: probed fct0
Apr 13 23:52:33 fio2 kernel: [  337.817645] fioinf ioMemory SX300-1300 0000:00:04.0: sector_size=4096
Apr 13 23:52:33 fio2 kernel: [  337.818492] fioinf ioMemory SX300-1300 0000:00:04.0: Setting channel range data to [2 .. 1023]
Apr 13 23:52:34 fio2 kernel: [  338.465417] fioinf ioMemory SX300-1300 0000:00:04.0: Found metadata in EBs 617-559, loading...
Apr 13 23:52:35 fio2 kernel: [  339.384146] fioinf ioMemory SX300-1300 0000:00:04.0: Device has no Virtual Partitions.  Creating compatibility Virtual Partition fioa.
Apr 13 23:52:35 fio2 kernel: [  339.385377] fioinf ioMemory SX300-1300 0000:00:04.0: Created device of size 1249999998976 bytes with 305175781 sectors of 4096 bytes (204131054 mapped).
Apr 13 23:52:35 fio2 kernel: [  339.391294] fioinf ioMemory SX300-1300 0000:00:04.0: Creating block device fioa: major: 251 minor: 0 sector size: 4096...
Apr 13 23:52:35 fio2 kernel: [  339.393795]  fioa: fioa1
Apr 13 23:52:35 fio2 kernel: [  339.394289] fioinf ioMemory SX300-1300 0000:00:04.0: Exposing Virtual Partition fioa of size 1249999998976 bytes with 305175781 sectors of 4096 bytes (204131054 mapped).
Apr 13 23:52:35 fio2 kernel: [  339.395807] fioinf ioMemory SX300-1300 0000:00:04.0: Attach succeeded.
Apr 13 23:52:46 fio2 kernel: [  350.215585] EXT4-fs (fioa1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
Apr 13 23:53:21 fio2 kernel: [  385.618523] fioinf ioMemory SX300-1300 0000:00:04.0: Shutting down block device fioa: major: 251 minor: 0 sector size: 4096...
Apr 13 23:53:21 fio2 kernel: [  385.619824] fioinf ioMemory SX300-1300 0000:00:04.0: Groomer for data log is tearing down
Apr 13 23:53:21 fio2 kernel: [  385.620957] fioinf ioMemory SX300-1300 0000:00:04.0: Groomer for data log halted.
Apr 13 23:53:21 fio2 kernel: [  385.914967] fioinf ioMemory SX300-1300 0000:00:04.0: Writing metadata starting at EB  272 (465 free 554 used)
Apr 13 23:53:23 fio2 kernel: [  387.144240] fioinf ioDrive driver unloading...
Apr 13 23:53:23 fio2 kernel: [  387.145055] fioinf ioDrive driver unloaded...
snuf commented 2 years ago

yeah so 5.16 is still bust....

Apr 14 00:27:19 fio2 kernel: [  259.844102] RIP: 0010:linux_bdev_update_stats+0x11/0x2c0 [iomemory_vsl4]
snuf commented 2 years ago

Seems like something odd changed with the part stats which bork on 5.16. Though not on 5.15, nor 5.13. I've changed the intermediate gendisk and pointer that was assigned, which seems to have fixed the issue. Am testing on 5.15 and 5.16 now.

snuf commented 2 years ago

Should be fixed now, and is merged with main

w41l commented 2 years ago

I can confirm the driver works as expected on Linux 5.15.37

[  +0.993179] fioinf ioDrive 0000:11:00.0: Product UUID is 0fd50a4d-8ab4-5bca-b5f5-ae5b552fd812
[  +0.000021] fioinf ioDrive 0000:11:00.0: Required PCIE bandwidth 4.000 GBytes per sec
[  +0.000010] fioinf ioDrive 0000:11:00.0: Board serial number is 1447G0216
[  +0.000006] fioinf ioDrive 0000:11:00.0: Adapter serial number is 1447G0216
[  +0.000011] fioinf ioDrive 0000:11:00.0: Default capacity        3200.000 GBytes
[  +0.000008] fioinf ioDrive 0000:11:00.0: Default sector size     4096 bytes
[  +0.000005] fioinf ioDrive 0000:11:00.0: Rated endurance         11.00 PBytes
[  +0.000008] fioinf ioDrive 0000:11:00.0: 85C temp range hardware found
[  +0.000007] fioinf ioDrive 0000:11:00.0: Maximum capacity        3200.000 GBytes
[  +0.000040] fioinf ioDrive 0000:11:00.0: PCIe Slot reported power limit: 75000mWatts
[  +0.000019] fioinf ioDrive 0000:11:00.0: PCIe Adapter power limit: 75000mWatts
[  +0.000006] fioinf ioDrive 0000:11:00.0: PCIe Adapter power Throttle point: 74750mWatts
[  +0.000013] fioinf ioDrive 0000:11:00.0: Firmware Archive Information:
[  +0.000007] fioinf ioDrive 0000:11:00.0:   * Name        = ioMemory Firmware 8.9.8.20161119
[  +0.000007] fioinf ioDrive 0000:11:00.0:   * Description = ioMemory Firmware 8.9.8.20161119 Archive
[  +0.000007] fioinf ioDrive 0000:11:00.0:   * Version     = 8.9.8.0
[  +0.000007] fioinf ioDrive 0000:11:00.0:   * Date        = 11/19/2016  Rev 20161119
[  +0.000009] fioinf ioDrive 0000:11:00.0: Firmware version 8.9.8 118189 (0x802408 0x1cdad)
[  +0.000009] fioinf ioDrive 0000:11:00.0: Platform version 41 
[  +0.000006] fioinf ioDrive 0000:11:00.0: Firmware VCS version 118189 [0x1cdad]
[  +0.000011] fioinf ioDrive 0000:11:00.0: Firmware VCS uid 0xa8aa40fe86ad1122411352667b9dc702a7c235af
[  +0.009571] fioinf ioDrive 0000:11:00.0: Powercut flush: Supported and Enabled.
[  +0.186104] fioinf ioDrive 0000:11:00.0: Loading microcode image 1.0.8 rev 100749 with flags 0x00010100
[  +0.000080] fioinf ioDrive 0000:11:00.0: Microcode loaded (0).
[  +0.078390] fioinf ioDrive 0000:11:00.0: Multiple queues enabled
[  +0.033351] fioinf ioDrive 0000:11:00.0: PCIe power monitor enabled (master). Limit set to 74.750 watts.
[  +0.000015] fioinf ioDrive 0000:11:00.0: Thermal monitoring: Enabled
[  +0.000004] fioinf ioDrive 0000:11:00.0: Hardware temperature alarm set for 85C.
[  +0.000908] lpfc 0000:51:00.1: 1:1303 Link Up Event x1 received Data: x1 x0 x80 x0 x0 x0 0
[  +0.101480] fioinf ioDrive 0000:11:00.0: Starting device IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0
[  +0.606386] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: probed fct0
[  +0.000327] iomemory-vsl4 0000:41:00.0: enabling device (0140 -> 0142)
[  +0.000379] fioinf ioDrive 0000:41:00.0: mapping controller on BAR 5
[  +0.110249] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: sector_size=4096
[  +0.002636] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Setting channel range data to [2 .. 1023]
[  +0.037423] fioinf ioDrive 0000:41:00.0: Configuring controller.
[  +0.000654] fioinf ioDrive 0000:41:00.0: using MSI-X interrupts
[  +0.000724] fioinf ioDrive 0000:41:00.0: Starting controller
[  +0.635996] fioinf ioDrive 0000:41:00.0: PMP Address: 1 1 1
[  +0.334875] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Found metadata in EBs 16-654, loading...
[  +1.357132] fioinf ioDrive 0000:41:00.0: SMP       fpga Microcode BOOT version 0.8.0 1
[  +0.000022] fioinf ioDrive 0000:41:00.0: SMP controller Microcode APP  version 0.15.0 0
[  +0.360702] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Device has no Virtual Partitions.  Creating compatibility Virtual Partition fioa.
[  +0.000057] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Created device of size 3200000000000 bytes with 781250000 sectors of 4096 bytes (3620004 mapped).
[  +0.002169] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Creating block device fioa: major: 251 minor: 0 sector size: 4096...
[  +0.000895] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Exposing Virtual Partition fioa of size 3200000000000 bytes with 781250000 sectors of 4096 bytes (3620004 mapped).
[  +0.000024] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:11:00.0: Attach succeeded.
[  +0.788197] fioinf ioDrive 0000:41:00.0: Product UUID is 93b97521-c82c-54d3-a55e-bd37411256aa
[  +0.000034] fioinf ioDrive 0000:41:00.0: Required PCIE bandwidth 4.000 GBytes per sec
[  +0.000020] fioinf ioDrive 0000:41:00.0: Board serial number is 1447G0068
[  +0.000006] fioinf ioDrive 0000:41:00.0: Adapter serial number is 1447G0068
[  +0.000010] fioinf ioDrive 0000:41:00.0: Default capacity        3200.000 GBytes
[  +0.000008] fioinf ioDrive 0000:41:00.0: Default sector size     4096 bytes
[  +0.000006] fioinf ioDrive 0000:41:00.0: Rated endurance         11.00 PBytes
[  +0.000006] fioinf ioDrive 0000:41:00.0: 85C temp range hardware found
[  +0.000008] fioinf ioDrive 0000:41:00.0: Maximum capacity        3200.000 GBytes
[  +0.000037] fioinf ioDrive 0000:41:00.0: PCIe Slot reported power limit: 75000mWatts
[  +0.000007] fioinf ioDrive 0000:41:00.0: PCIe Adapter power limit: 75000mWatts
[  +0.000007] fioinf ioDrive 0000:41:00.0: PCIe Adapter power Throttle point: 74750mWatts
[  +0.000012] fioinf ioDrive 0000:41:00.0: Firmware Archive Information:
[  +0.000007] fioinf ioDrive 0000:41:00.0:   * Name        = ioMemory Firmware 8.9.8.20161119
[  +0.000007] fioinf ioDrive 0000:41:00.0:   * Description = ioMemory Firmware 8.9.8.20161119 Archive
[  +0.000007] fioinf ioDrive 0000:41:00.0:   * Version     = 8.9.8.0
[  +0.000007] fioinf ioDrive 0000:41:00.0:   * Date        = 11/19/2016  Rev 20161119
[  +0.000009] fioinf ioDrive 0000:41:00.0: Firmware version 8.9.8 118189 (0x802408 0x1cdad)
[  +0.000009] fioinf ioDrive 0000:41:00.0: Platform version 41 
[  +0.000006] fioinf ioDrive 0000:41:00.0: Firmware VCS version 118189 [0x1cdad]
[  +0.000010] fioinf ioDrive 0000:41:00.0: Firmware VCS uid 0xa8aa40fe86ad1122411352667b9dc702a7c235af
[  +0.018808] fioinf ioDrive 0000:41:00.0: Powercut flush: Supported and Enabled.
[  +0.196803] fioinf ioDrive 0000:41:00.0: Loading microcode image 1.0.8 rev 100749 with flags 0x00010100
[  +0.000080] fioinf ioDrive 0000:41:00.0: Microcode loaded (0).
[  +0.078487] fioinf ioDrive 0000:41:00.0: Multiple queues enabled
[  +0.033364] fioinf ioDrive 0000:41:00.0: PCIe power monitor enabled (master). Limit set to 74.750 watts.
[  +0.000013] fioinf ioDrive 0000:41:00.0: Thermal monitoring: Enabled
[  +0.000004] fioinf ioDrive 0000:41:00.0: Hardware temperature alarm set for 85C.
[  +0.093540] fioinf ioDrive 0000:41:00.0: Starting device IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0
[  +0.735617] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: probed fct1
[  +0.162683] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: sector_size=4096
[  +0.003179] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Setting channel range data to [2 .. 1023]
[  +0.962630] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Found metadata in EBs 528-419, loading...
[  +1.882862] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Device has no Virtual Partitions.  Creating compatibility Virtual Partition fiob.
[  +0.000057] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Created device of size 3200000000000 bytes with 781250000 sectors of 4096 bytes (1701028 mapped).
[  +0.000616] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Creating block device fiob: major: 251 minor: 16 sector size: 4096...
[  +0.000755] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Exposing Virtual Partition fiob of size 3200000000000 bytes with 781250000 sectors of 4096 bytes (1701028 mapped).
[  +0.000011] fioinf IBM 3200GB Enterprise Value io3 Flash Adapter for System x 0000:41:00.0: Attach succeeded.

Thank you