search

spdk / spdk

Storage Performance Development Kit
https://spdk.io/
Other
3.1k stars 1.21k forks source link

Some SPDK components are not installed during 'make install' #3143

Closed benlwalker closed 9 months ago

benlwalker commented 1 year ago

Sighting report

Most of us use SPDK that we built ourselves and we don't bother to install it. However, when building SPDK and installing it onto a system several things show up very quickly:

1) If you are building shared libraries and you used the DPDK submodule, DPDK does not get installed, so all of the SPDK applications won't run. 2) Same as 1, but for ISA-L. 3) The fio plugins don't get installed 4) bdevperf doesn't get installed (should we promote this to an official tool and put it into /bin?)

Expected Behavior

I don't know. If you are building SPDK with shared libraries against the DPDK submodule, DPDK needs to get installed on the system somehow. But what if the system already has DPDK installed?

The fio plugins seem like they should get installed, but where do we put them?

For bdevperf, if we decide it's time to promote this to an official tool, then we need to rename it to spdk_bdev_perf and put it into bin/ like everything else. This is just a decision to make - the mechanics of this one are easy.

Current Behavior

These things don't get installed.

Possible Solution

Maybe it shouldn't automatically install DPDK, but instead add a special make install dpdk? Maybe the fio plugins just go into the library install path?

Steps to Reproduce

./configure --with-shared --with-fio=<> && make && make install

Context (Environment including OS version, SPDK version, etc.)

Pretend you're an end-user on a fresh Linux system and building a software package provided as source to be included into a container or to a system provided to other users.

mikeBashStuff commented 1 year ago

Just a note, food for thought. :)

Our packaging (rpmbuild) is sort of covering this since, for instance, we build an extra package with dpdk libs inside (some other, like isa-l's, are explicitly included in the main lib package) and we do have tests which do cover that.

That said, the scope is minimal as we don't build these rpms with "full" SPDK inside. But the use-case like --with-shared --with-fio can definitely work. Also, our docker/* stuff also depends on these RPMs.

So essentially this:

./rpmbuild/rpm.sh --with-shared --with-fio
rpm -i "$HOME/rpmbuild/rpm/"*/spdk-*

would deploy a working SPDK base under your distro (rpm-like that is, though there's nothing wrong with extracting these packages under any other non-rpm distro). Dirty view of what's get installed from the example above:


[root@test-hostname ~]# rpm -qa | grep spdk
spdk-libs-v24.01-1.x86_64
spdk-dpdk-libs-v24.01-1.x86_64
spdk-v24.01-1.x86_64
spdk-devel-v24.01-1.x86_64
[root@test-hostname ~]# rpm -ql $(rpm -qa | grep spdk) | grep -v build-id
/etc/ld.so.conf.d/spdk.conf
/usr/local/lib/libspdk.so.5.0
/usr/local/lib/libspdk_accel.a
/usr/local/lib/libspdk_accel.so.13.0
/usr/local/lib/libspdk_accel_ioat.a
/usr/local/lib/libspdk_accel_ioat.so.5.0
/usr/local/lib/libspdk_bdev.a
/usr/local/lib/libspdk_bdev.so.13.0
/usr/local/lib/libspdk_bdev_aio.a
/usr/local/lib/libspdk_bdev_aio.so.5.0
/usr/local/lib/libspdk_bdev_delay.a
/usr/local/lib/libspdk_bdev_delay.so.5.0
/usr/local/lib/libspdk_bdev_error.a
/usr/local/lib/libspdk_bdev_error.so.5.0
/usr/local/lib/libspdk_bdev_ftl.a
/usr/local/lib/libspdk_bdev_ftl.so.5.0
/usr/local/lib/libspdk_bdev_gpt.a
/usr/local/lib/libspdk_bdev_gpt.so.5.0
/usr/local/lib/libspdk_bdev_lvol.a
/usr/local/lib/libspdk_bdev_lvol.so.5.0
/usr/local/lib/libspdk_bdev_malloc.a
/usr/local/lib/libspdk_bdev_malloc.so.5.0
/usr/local/lib/libspdk_bdev_null.a
/usr/local/lib/libspdk_bdev_null.so.5.0
/usr/local/lib/libspdk_bdev_nvme.a
/usr/local/lib/libspdk_bdev_nvme.so.6.0
/usr/local/lib/libspdk_bdev_passthru.a
/usr/local/lib/libspdk_bdev_passthru.so.5.0
/usr/local/lib/libspdk_bdev_raid.a
/usr/local/lib/libspdk_bdev_raid.so.5.0
/usr/local/lib/libspdk_bdev_split.a
/usr/local/lib/libspdk_bdev_split.so.5.0
/usr/local/lib/libspdk_bdev_virtio.a
/usr/local/lib/libspdk_bdev_virtio.so.5.0
/usr/local/lib/libspdk_bdev_zone_block.a
/usr/local/lib/libspdk_bdev_zone_block.so.5.0
/usr/local/lib/libspdk_blob.a
/usr/local/lib/libspdk_blob.so.10.0
/usr/local/lib/libspdk_blob_bdev.a
/usr/local/lib/libspdk_blob_bdev.so.10.0
/usr/local/lib/libspdk_blobfs.a
/usr/local/lib/libspdk_blobfs.so.9.0
/usr/local/lib/libspdk_blobfs_bdev.a
/usr/local/lib/libspdk_blobfs_bdev.so.5.0
/usr/local/lib/libspdk_conf.a
/usr/local/lib/libspdk_conf.so.5.0
/usr/local/lib/libspdk_dma.a
/usr/local/lib/libspdk_dma.so.3.0
/usr/local/lib/libspdk_env_dpdk.a
/usr/local/lib/libspdk_env_dpdk.so.12.0
/usr/local/lib/libspdk_env_dpdk_rpc.a
/usr/local/lib/libspdk_env_dpdk_rpc.so.5.0
/usr/local/lib/libspdk_event.a
/usr/local/lib/libspdk_event.so.12.0
/usr/local/lib/libspdk_event_accel.a
/usr/local/lib/libspdk_event_accel.so.5.0
/usr/local/lib/libspdk_event_bdev.a
/usr/local/lib/libspdk_event_bdev.so.5.0
/usr/local/lib/libspdk_event_iobuf.a
/usr/local/lib/libspdk_event_iobuf.so.2.0
/usr/local/lib/libspdk_event_iscsi.a
/usr/local/lib/libspdk_event_iscsi.so.5.0
/usr/local/lib/libspdk_event_nbd.a
/usr/local/lib/libspdk_event_nbd.so.5.0
/usr/local/lib/libspdk_event_nvmf.a
/usr/local/lib/libspdk_event_nvmf.so.5.0
/usr/local/lib/libspdk_event_scheduler.a
/usr/local/lib/libspdk_event_scheduler.so.3.0
/usr/local/lib/libspdk_event_scsi.a
/usr/local/lib/libspdk_event_scsi.so.5.0
/usr/local/lib/libspdk_event_sock.a
/usr/local/lib/libspdk_event_sock.so.4.0
/usr/local/lib/libspdk_event_vhost_blk.a
/usr/local/lib/libspdk_event_vhost_blk.so.2.0
/usr/local/lib/libspdk_event_vhost_scsi.a
/usr/local/lib/libspdk_event_vhost_scsi.so.2.0
/usr/local/lib/libspdk_event_vmd.a
/usr/local/lib/libspdk_event_vmd.so.5.0
/usr/local/lib/libspdk_ftl.a
/usr/local/lib/libspdk_ftl.so.7.0
/usr/local/lib/libspdk_init.a
/usr/local/lib/libspdk_init.so.4.0
/usr/local/lib/libspdk_ioat.a
/usr/local/lib/libspdk_ioat.so.6.0
/usr/local/lib/libspdk_iscsi.a
/usr/local/lib/libspdk_iscsi.so.7.0
/usr/local/lib/libspdk_json.a
/usr/local/lib/libspdk_json.so.5.0
/usr/local/lib/libspdk_jsonrpc.a
/usr/local/lib/libspdk_jsonrpc.so.5.1
/usr/local/lib/libspdk_log.a
/usr/local/lib/libspdk_log.so.6.1
/usr/local/lib/libspdk_lvol.a
/usr/local/lib/libspdk_lvol.so.9.0
/usr/local/lib/libspdk_nbd.a
/usr/local/lib/libspdk_nbd.so.6.0
/usr/local/lib/libspdk_notify.a
/usr/local/lib/libspdk_notify.so.5.0
/usr/local/lib/libspdk_nvme.a
/usr/local/lib/libspdk_nvme.so.11.0
/usr/local/lib/libspdk_nvmf.a
/usr/local/lib/libspdk_nvmf.so.16.0
/usr/local/lib/libspdk_rpc.a
/usr/local/lib/libspdk_rpc.so.5.0
/usr/local/lib/libspdk_scheduler_dpdk_governor.a
/usr/local/lib/libspdk_scheduler_dpdk_governor.so.3.0
/usr/local/lib/libspdk_scheduler_dynamic.a
/usr/local/lib/libspdk_scheduler_dynamic.so.3.0
/usr/local/lib/libspdk_scheduler_gscheduler.a
/usr/local/lib/libspdk_scheduler_gscheduler.so.3.0
/usr/local/lib/libspdk_scsi.a
/usr/local/lib/libspdk_scsi.so.7.0
/usr/local/lib/libspdk_sock.a
/usr/local/lib/libspdk_sock.so.8.0
/usr/local/lib/libspdk_sock_posix.a
/usr/local/lib/libspdk_sock_posix.so.5.0
/usr/local/lib/libspdk_thread.a
/usr/local/lib/libspdk_thread.so.8.1
/usr/local/lib/libspdk_trace.a
/usr/local/lib/libspdk_trace.so.8.0
/usr/local/lib/libspdk_trace_parser.a
/usr/local/lib/libspdk_trace_parser.so.4.0
/usr/local/lib/libspdk_ut_mock.a
/usr/local/lib/libspdk_ut_mock.so.5.0
/usr/local/lib/libspdk_util.a
/usr/local/lib/libspdk_util.so.7.0
/usr/local/lib/libspdk_vfio_user.a
/usr/local/lib/libspdk_vfio_user.so.4.0
/usr/local/lib/libspdk_vhost.a
/usr/local/lib/libspdk_vhost.so.7.0
/usr/local/lib/libspdk_virtio.a
/usr/local/lib/libspdk_virtio.so.6.0
/usr/local/lib/libspdk_vmd.a
/usr/local/lib/libspdk_vmd.so.5.0
/usr/local/lib/pkgconfig/spdk_accel.pc
/usr/local/lib/pkgconfig/spdk_accel_ioat.pc
/usr/local/lib/pkgconfig/spdk_accel_modules.pc
/usr/local/lib/pkgconfig/spdk_bdev.pc
/usr/local/lib/pkgconfig/spdk_bdev_aio.pc
/usr/local/lib/pkgconfig/spdk_bdev_delay.pc
/usr/local/lib/pkgconfig/spdk_bdev_error.pc
/usr/local/lib/pkgconfig/spdk_bdev_ftl.pc
/usr/local/lib/pkgconfig/spdk_bdev_gpt.pc
/usr/local/lib/pkgconfig/spdk_bdev_lvol.pc
/usr/local/lib/pkgconfig/spdk_bdev_malloc.pc
/usr/local/lib/pkgconfig/spdk_bdev_modules.pc
/usr/local/lib/pkgconfig/spdk_bdev_null.pc
/usr/local/lib/pkgconfig/spdk_bdev_nvme.pc
/usr/local/lib/pkgconfig/spdk_bdev_passthru.pc
/usr/local/lib/pkgconfig/spdk_bdev_raid.pc
/usr/local/lib/pkgconfig/spdk_bdev_split.pc
/usr/local/lib/pkgconfig/spdk_bdev_virtio.pc
/usr/local/lib/pkgconfig/spdk_bdev_zone_block.pc
/usr/local/lib/pkgconfig/spdk_blob.pc
/usr/local/lib/pkgconfig/spdk_blob_bdev.pc
/usr/local/lib/pkgconfig/spdk_blobfs.pc
/usr/local/lib/pkgconfig/spdk_blobfs_bdev.pc
/usr/local/lib/pkgconfig/spdk_conf.pc
/usr/local/lib/pkgconfig/spdk_dma.pc
/usr/local/lib/pkgconfig/spdk_dpdklibs.pc
/usr/local/lib/pkgconfig/spdk_env_dpdk.pc
/usr/local/lib/pkgconfig/spdk_env_dpdk_rpc.pc
/usr/local/lib/pkgconfig/spdk_event.pc
/usr/local/lib/pkgconfig/spdk_event_accel.pc
/usr/local/lib/pkgconfig/spdk_event_bdev.pc
/usr/local/lib/pkgconfig/spdk_event_iobuf.pc
/usr/local/lib/pkgconfig/spdk_event_iscsi.pc
/usr/local/lib/pkgconfig/spdk_event_nbd.pc
/usr/local/lib/pkgconfig/spdk_event_nvmf.pc
/usr/local/lib/pkgconfig/spdk_event_scheduler.pc
/usr/local/lib/pkgconfig/spdk_event_scsi.pc
/usr/local/lib/pkgconfig/spdk_event_sock.pc
/usr/local/lib/pkgconfig/spdk_event_vhost_blk.pc
/usr/local/lib/pkgconfig/spdk_event_vhost_scsi.pc
/usr/local/lib/pkgconfig/spdk_event_vmd.pc
/usr/local/lib/pkgconfig/spdk_ftl.pc
/usr/local/lib/pkgconfig/spdk_init.pc
/usr/local/lib/pkgconfig/spdk_ioat.pc
/usr/local/lib/pkgconfig/spdk_iscsi.pc
/usr/local/lib/pkgconfig/spdk_json.pc
/usr/local/lib/pkgconfig/spdk_jsonrpc.pc
/usr/local/lib/pkgconfig/spdk_log.pc
/usr/local/lib/pkgconfig/spdk_lvol.pc
/usr/local/lib/pkgconfig/spdk_nbd.pc
/usr/local/lib/pkgconfig/spdk_notify.pc
/usr/local/lib/pkgconfig/spdk_nvme.pc
/usr/local/lib/pkgconfig/spdk_nvmf.pc
/usr/local/lib/pkgconfig/spdk_rpc.pc
/usr/local/lib/pkgconfig/spdk_scheduler_dpdk_governor.pc
/usr/local/lib/pkgconfig/spdk_scheduler_dynamic.pc
/usr/local/lib/pkgconfig/spdk_scheduler_gscheduler.pc
/usr/local/lib/pkgconfig/spdk_scheduler_modules.pc
/usr/local/lib/pkgconfig/spdk_scsi.pc
/usr/local/lib/pkgconfig/spdk_sock.pc
/usr/local/lib/pkgconfig/spdk_sock_modules.pc
/usr/local/lib/pkgconfig/spdk_sock_posix.pc
/usr/local/lib/pkgconfig/spdk_syslibs.pc
/usr/local/lib/pkgconfig/spdk_thread.pc
/usr/local/lib/pkgconfig/spdk_trace.pc
/usr/local/lib/pkgconfig/spdk_trace_parser.pc
/usr/local/lib/pkgconfig/spdk_ut_mock.pc
/usr/local/lib/pkgconfig/spdk_util.pc
/usr/local/lib/pkgconfig/spdk_vfio_user.pc
/usr/local/lib/pkgconfig/spdk_vhost.pc
/usr/local/lib/pkgconfig/spdk_virtio.pc
/usr/local/lib/pkgconfig/spdk_vmd.pc
/usr/local/lib/dpdk
/usr/local/lib/dpdk/dpdk
/usr/local/lib/dpdk/dpdk/pmds-23.2
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_bus_pci.so
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_bus_pci.so.23
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_bus_pci.so.23.2
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_bus_vdev.so
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_bus_vdev.so.23
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_bus_vdev.so.23.2
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_mempool_ring.so
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_mempool_ring.so.23
/usr/local/lib/dpdk/dpdk/pmds-23.2/librte_mempool_ring.so.23.2
/usr/local/lib/dpdk/librte_bus_pci.a
/usr/local/lib/dpdk/librte_bus_pci.so
/usr/local/lib/dpdk/librte_bus_pci.so.23
/usr/local/lib/dpdk/librte_bus_pci.so.23.2
/usr/local/lib/dpdk/librte_bus_vdev.a
/usr/local/lib/dpdk/librte_bus_vdev.so
/usr/local/lib/dpdk/librte_bus_vdev.so.23
/usr/local/lib/dpdk/librte_bus_vdev.so.23.2
/usr/local/lib/dpdk/librte_cmdline.a
/usr/local/lib/dpdk/librte_cmdline.so
/usr/local/lib/dpdk/librte_cmdline.so.23
/usr/local/lib/dpdk/librte_cmdline.so.23.2
/usr/local/lib/dpdk/librte_compressdev.a
/usr/local/lib/dpdk/librte_compressdev.so
/usr/local/lib/dpdk/librte_compressdev.so.23
/usr/local/lib/dpdk/librte_compressdev.so.23.2
/usr/local/lib/dpdk/librte_cryptodev.a
/usr/local/lib/dpdk/librte_cryptodev.so
/usr/local/lib/dpdk/librte_cryptodev.so.23
/usr/local/lib/dpdk/librte_cryptodev.so.23.2
/usr/local/lib/dpdk/librte_dmadev.a
/usr/local/lib/dpdk/librte_dmadev.so
/usr/local/lib/dpdk/librte_dmadev.so.23
/usr/local/lib/dpdk/librte_dmadev.so.23.2
/usr/local/lib/dpdk/librte_eal.a
/usr/local/lib/dpdk/librte_eal.so
/usr/local/lib/dpdk/librte_eal.so.23
/usr/local/lib/dpdk/librte_eal.so.23.2
/usr/local/lib/dpdk/librte_ethdev.a
/usr/local/lib/dpdk/librte_ethdev.so
/usr/local/lib/dpdk/librte_ethdev.so.23
/usr/local/lib/dpdk/librte_ethdev.so.23.2
/usr/local/lib/dpdk/librte_hash.a
/usr/local/lib/dpdk/librte_hash.so
/usr/local/lib/dpdk/librte_hash.so.23
/usr/local/lib/dpdk/librte_hash.so.23.2
/usr/local/lib/dpdk/librte_kvargs.a
/usr/local/lib/dpdk/librte_kvargs.so
/usr/local/lib/dpdk/librte_kvargs.so.23
/usr/local/lib/dpdk/librte_kvargs.so.23.2
/usr/local/lib/dpdk/librte_mbuf.a
/usr/local/lib/dpdk/librte_mbuf.so
/usr/local/lib/dpdk/librte_mbuf.so.23
/usr/local/lib/dpdk/librte_mbuf.so.23.2
/usr/local/lib/dpdk/librte_mempool.a
/usr/local/lib/dpdk/librte_mempool.so
/usr/local/lib/dpdk/librte_mempool.so.23
/usr/local/lib/dpdk/librte_mempool.so.23.2
/usr/local/lib/dpdk/librte_mempool_ring.a
/usr/local/lib/dpdk/librte_mempool_ring.so
/usr/local/lib/dpdk/librte_mempool_ring.so.23
/usr/local/lib/dpdk/librte_mempool_ring.so.23.2
/usr/local/lib/dpdk/librte_meter.a
/usr/local/lib/dpdk/librte_meter.so
/usr/local/lib/dpdk/librte_meter.so.23
/usr/local/lib/dpdk/librte_meter.so.23.2
/usr/local/lib/dpdk/librte_net.a
/usr/local/lib/dpdk/librte_net.so
/usr/local/lib/dpdk/librte_net.so.23
/usr/local/lib/dpdk/librte_net.so.23.2
/usr/local/lib/dpdk/librte_pci.a
/usr/local/lib/dpdk/librte_pci.so
/usr/local/lib/dpdk/librte_pci.so.23
/usr/local/lib/dpdk/librte_pci.so.23.2
/usr/local/lib/dpdk/librte_power.a
/usr/local/lib/dpdk/librte_power.so
/usr/local/lib/dpdk/librte_power.so.23
/usr/local/lib/dpdk/librte_power.so.23.2
/usr/local/lib/dpdk/librte_rcu.a
/usr/local/lib/dpdk/librte_rcu.so
/usr/local/lib/dpdk/librte_rcu.so.23
/usr/local/lib/dpdk/librte_rcu.so.23.2
/usr/local/lib/dpdk/librte_reorder.a
/usr/local/lib/dpdk/librte_reorder.so
/usr/local/lib/dpdk/librte_reorder.so.23
/usr/local/lib/dpdk/librte_reorder.so.23.2
/usr/local/lib/dpdk/librte_ring.a
/usr/local/lib/dpdk/librte_ring.so
/usr/local/lib/dpdk/librte_ring.so.23
/usr/local/lib/dpdk/librte_ring.so.23.2
/usr/local/lib/dpdk/librte_security.a
/usr/local/lib/dpdk/librte_security.so
/usr/local/lib/dpdk/librte_security.so.23
/usr/local/lib/dpdk/librte_security.so.23.2
/usr/local/lib/dpdk/librte_telemetry.a
/usr/local/lib/dpdk/librte_telemetry.so
/usr/local/lib/dpdk/librte_telemetry.so.23
/usr/local/lib/dpdk/librte_telemetry.so.23.2
/usr/local/lib/dpdk/librte_timer.a
/usr/local/lib/dpdk/librte_timer.so
/usr/local/lib/dpdk/librte_timer.so.23
/usr/local/lib/dpdk/librte_timer.so.23.2
/usr/local/lib/dpdk/librte_vhost.a
/usr/local/lib/dpdk/librte_vhost.so
/usr/local/lib/dpdk/librte_vhost.so.23
/usr/local/lib/dpdk/librte_vhost.so.23.2
/usr/local/lib/dpdk/pkgconfig
/usr/local/lib/dpdk/pkgconfig/libdpdk-libs.pc
/usr/local/lib/dpdk/pkgconfig/libdpdk.pc
/etc/bash_completion.d/spdk
/etc/profile.d/spdk_path.sh
/usr/lib/python3.11/site-packages/spdk-24.1rc0-py3.11.egg-info/PKG-INFO
/usr/lib/python3.11/site-packages/spdk-24.1rc0-py3.11.egg-info/SOURCES.txt
/usr/lib/python3.11/site-packages/spdk-24.1rc0-py3.11.egg-info/dependency_links.txt
/usr/lib/python3.11/site-packages/spdk-24.1rc0-py3.11.egg-info/top_level.txt
/usr/lib/python3.11/site-packages/spdk/__init__.py
/usr/lib/python3.11/site-packages/spdk/__pycache__
/usr/lib/python3.11/site-packages/spdk/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc
/usr/lib/python3.11/site-packages/spdk/rpc/__init__.py
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/accel.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/app.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/bdev.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/blobfs.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/client.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/cmd_parser.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/compressdev.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/dpdk_cryptodev.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/dsa.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/env_dpdk.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/helpers.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/iaa.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/ioat.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/iobuf.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/iscsi.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/log.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/lvol.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/mlx5.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/nbd.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/notify.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/nvme.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/nvmf.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/sock.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/subsystem.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/trace.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/ublk.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/vfio_user.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/vhost.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/__pycache__/vmd.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/rpc/accel.py
/usr/lib/python3.11/site-packages/spdk/rpc/app.py
/usr/lib/python3.11/site-packages/spdk/rpc/bdev.py
/usr/lib/python3.11/site-packages/spdk/rpc/blobfs.py
/usr/lib/python3.11/site-packages/spdk/rpc/client.py
/usr/lib/python3.11/site-packages/spdk/rpc/cmd_parser.py
/usr/lib/python3.11/site-packages/spdk/rpc/compressdev.py
/usr/lib/python3.11/site-packages/spdk/rpc/dpdk_cryptodev.py
/usr/lib/python3.11/site-packages/spdk/rpc/dsa.py
/usr/lib/python3.11/site-packages/spdk/rpc/env_dpdk.py
/usr/lib/python3.11/site-packages/spdk/rpc/helpers.py
/usr/lib/python3.11/site-packages/spdk/rpc/iaa.py
/usr/lib/python3.11/site-packages/spdk/rpc/ioat.py
/usr/lib/python3.11/site-packages/spdk/rpc/iobuf.py
/usr/lib/python3.11/site-packages/spdk/rpc/iscsi.py
/usr/lib/python3.11/site-packages/spdk/rpc/log.py
/usr/lib/python3.11/site-packages/spdk/rpc/lvol.py
/usr/lib/python3.11/site-packages/spdk/rpc/mlx5.py
/usr/lib/python3.11/site-packages/spdk/rpc/nbd.py
/usr/lib/python3.11/site-packages/spdk/rpc/notify.py
/usr/lib/python3.11/site-packages/spdk/rpc/nvme.py
/usr/lib/python3.11/site-packages/spdk/rpc/nvmf.py
/usr/lib/python3.11/site-packages/spdk/rpc/sock.py
/usr/lib/python3.11/site-packages/spdk/rpc/subsystem.py
/usr/lib/python3.11/site-packages/spdk/rpc/trace.py
/usr/lib/python3.11/site-packages/spdk/rpc/ublk.py
/usr/lib/python3.11/site-packages/spdk/rpc/vfio_user.py
/usr/lib/python3.11/site-packages/spdk/rpc/vhost.py
/usr/lib/python3.11/site-packages/spdk/rpc/vmd.py
/usr/lib/python3.11/site-packages/spdk/sma
/usr/lib/python3.11/site-packages/spdk/sma/__init__.py
/usr/lib/python3.11/site-packages/spdk/sma/__pycache__
/usr/lib/python3.11/site-packages/spdk/sma/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/__pycache__/common.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/__pycache__/qmp.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/__pycache__/qos.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/__pycache__/sma.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/common.py
/usr/lib/python3.11/site-packages/spdk/sma/device
/usr/lib/python3.11/site-packages/spdk/sma/device/__init__.py
/usr/lib/python3.11/site-packages/spdk/sma/device/__pycache__
/usr/lib/python3.11/site-packages/spdk/sma/device/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/device/__pycache__/device.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/device/__pycache__/nvmf_tcp.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/device/__pycache__/nvmf_vfiouser.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/device/__pycache__/vhost_blk.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/device/device.py
/usr/lib/python3.11/site-packages/spdk/sma/device/nvmf_tcp.py
/usr/lib/python3.11/site-packages/spdk/sma/device/nvmf_vfiouser.py
/usr/lib/python3.11/site-packages/spdk/sma/device/vhost_blk.py
/usr/lib/python3.11/site-packages/spdk/sma/proto
/usr/lib/python3.11/site-packages/spdk/sma/proto/__init__.py
/usr/lib/python3.11/site-packages/spdk/sma/proto/__pycache__
/usr/lib/python3.11/site-packages/spdk/sma/proto/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/qmp.py
/usr/lib/python3.11/site-packages/spdk/sma/qos.py
/usr/lib/python3.11/site-packages/spdk/sma/sma.py
/usr/lib/python3.11/site-packages/spdk/sma/volume
/usr/lib/python3.11/site-packages/spdk/sma/volume/__init__.py
/usr/lib/python3.11/site-packages/spdk/sma/volume/__pycache__
/usr/lib/python3.11/site-packages/spdk/sma/volume/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/volume/__pycache__/crypto.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/volume/__pycache__/crypto_bdev.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/volume/__pycache__/volume.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/sma/volume/crypto.py
/usr/lib/python3.11/site-packages/spdk/sma/volume/crypto_bdev.py
/usr/lib/python3.11/site-packages/spdk/sma/volume/volume.py
/usr/lib/python3.11/site-packages/spdk/spdkcli
/usr/lib/python3.11/site-packages/spdk/spdkcli/__init__.py
/usr/lib/python3.11/site-packages/spdk/spdkcli/__pycache__
/usr/lib/python3.11/site-packages/spdk/spdkcli/__pycache__/__init__.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/spdkcli/__pycache__/ui_node.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/spdkcli/__pycache__/ui_node_iscsi.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/spdkcli/__pycache__/ui_node_nvmf.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/spdkcli/__pycache__/ui_root.cpython-311.pyc
/usr/lib/python3.11/site-packages/spdk/spdkcli/ui_node.py
/usr/lib/python3.11/site-packages/spdk/spdkcli/ui_node_iscsi.py
/usr/lib/python3.11/site-packages/spdk/spdkcli/ui_node_nvmf.py
/usr/lib/python3.11/site-packages/spdk/spdkcli/ui_root.py
/usr/libexec/spdk/include
/usr/libexec/spdk/scripts
/usr/libexec/spdk/scripts/ar-xnvme-fixer
/usr/libexec/spdk/scripts/arm_cross_compile.sh
/usr/libexec/spdk/scripts/backport.sh
/usr/libexec/spdk/scripts/bash-completion
/usr/libexec/spdk/scripts/bash-completion/spdk
/usr/libexec/spdk/scripts/bpf
/usr/libexec/spdk/scripts/bpf/gen.py
/usr/libexec/spdk/scripts/bpf/gen_enums.sh
/usr/libexec/spdk/scripts/bpf/intr-wakeups.bt
/usr/libexec/spdk/scripts/bpf/nvme.bt
/usr/libexec/spdk/scripts/bpf/nvmf.bt
/usr/libexec/spdk/scripts/bpf/nvmf_path.bt
/usr/libexec/spdk/scripts/bpf/nvmf_timeout.bt
/usr/libexec/spdk/scripts/bpf/readv.bt
/usr/libexec/spdk/scripts/bpf/sched.bt
/usr/libexec/spdk/scripts/bpf/send_msg.bt
/usr/libexec/spdk/scripts/bpf/syscalls.bt
/usr/libexec/spdk/scripts/bpf/trace.py
/usr/libexec/spdk/scripts/bpftrace.sh
/usr/libexec/spdk/scripts/calc-iobuf.py
/usr/libexec/spdk/scripts/ceph
/usr/libexec/spdk/scripts/ceph/ceph.conf
/usr/libexec/spdk/scripts/ceph/start.sh
/usr/libexec/spdk/scripts/ceph/stop.sh
/usr/libexec/spdk/scripts/check_format.sh
/usr/libexec/spdk/scripts/common
/usr/libexec/spdk/scripts/common.sh
/usr/libexec/spdk/scripts/common/setup
/usr/libexec/spdk/scripts/common/setup/interactive.sh
/usr/libexec/spdk/scripts/core-collector.sh
/usr/libexec/spdk/scripts/detect_cc.sh
/usr/libexec/spdk/scripts/dpdk_mem_info.py
/usr/libexec/spdk/scripts/env_dpdk
/usr/libexec/spdk/scripts/env_dpdk/22.07
/usr/libexec/spdk/scripts/env_dpdk/22.07/22.07-rte_bus_pci.h.patch
/usr/libexec/spdk/scripts/env_dpdk/22.11
/usr/libexec/spdk/scripts/env_dpdk/22.11/23.07-rte_bus_pci.h.patch
/usr/libexec/spdk/scripts/env_dpdk/22.11/23.11-rte_dev.h.patch
/usr/libexec/spdk/scripts/env_dpdk/README.md
/usr/libexec/spdk/scripts/env_dpdk/check_dpdk_pci_api.sh
/usr/libexec/spdk/scripts/eofnl
/usr/libexec/spdk/scripts/fio-wrapper
/usr/libexec/spdk/scripts/gdb_macros.py
/usr/libexec/spdk/scripts/gen_ftl.sh
/usr/libexec/spdk/scripts/gen_nvme.sh
/usr/libexec/spdk/scripts/gen_sma_goapi.sh
/usr/libexec/spdk/scripts/genconfig.py
/usr/libexec/spdk/scripts/get-pmr
/usr/libexec/spdk/scripts/histogram.py
/usr/libexec/spdk/scripts/iostat.py
/usr/libexec/spdk/scripts/ledctl.sh
/usr/libexec/spdk/scripts/lspci
/usr/libexec/spdk/scripts/pc.sh
/usr/libexec/spdk/scripts/pc_libs.sh
/usr/libexec/spdk/scripts/pc_modules.sh
/usr/libexec/spdk/scripts/perf
/usr/libexec/spdk/scripts/perf/nvme
/usr/libexec/spdk/scripts/perf/nvme/README
/usr/libexec/spdk/scripts/perf/nvme/fio_test.conf
/usr/libexec/spdk/scripts/perf/nvme/run_fio_test.py
/usr/libexec/spdk/scripts/perf/nvme/run_fio_test.sh
/usr/libexec/spdk/scripts/perf/nvmf
/usr/libexec/spdk/scripts/perf/nvmf/README.md
/usr/libexec/spdk/scripts/perf/nvmf/common.py
/usr/libexec/spdk/scripts/perf/nvmf/config.json
/usr/libexec/spdk/scripts/perf/nvmf/run_nvmf.py
/usr/libexec/spdk/scripts/perf/nvmf/set_xps_rxqs
/usr/libexec/spdk/scripts/perf/pm
/usr/libexec/spdk/scripts/perf/pm/collect-bmc-pm
/usr/libexec/spdk/scripts/perf/vhost
/usr/libexec/spdk/scripts/perf/vhost/conf-generator
/usr/libexec/spdk/scripts/perf/vhost/run_vhost_test.sh
/usr/libexec/spdk/scripts/pkgdep
/usr/libexec/spdk/scripts/pkgdep.sh
/usr/libexec/spdk/scripts/pkgdep/arch.sh
/usr/libexec/spdk/scripts/pkgdep/centos.sh
/usr/libexec/spdk/scripts/pkgdep/common.sh
/usr/libexec/spdk/scripts/pkgdep/debian.sh
/usr/libexec/spdk/scripts/pkgdep/fedora.sh
/usr/libexec/spdk/scripts/pkgdep/freebsd.sh
/usr/libexec/spdk/scripts/pkgdep/openeuler.sh
/usr/libexec/spdk/scripts/pkgdep/requirements.txt
/usr/libexec/spdk/scripts/pkgdep/rhel.sh
/usr/libexec/spdk/scripts/pkgdep/sles.sh
/usr/libexec/spdk/scripts/pkgdep/ubuntu.sh
/usr/libexec/spdk/scripts/posix.txt
/usr/libexec/spdk/scripts/prep_benchmarks.sh
/usr/libexec/spdk/scripts/qat_setup.sh
/usr/libexec/spdk/scripts/rpc.py
/usr/libexec/spdk/scripts/rpc_http_proxy.py
/usr/libexec/spdk/scripts/rxe_cfg_small.sh
/usr/libexec/spdk/scripts/setup.sh
/usr/libexec/spdk/scripts/sma-client.py
/usr/libexec/spdk/scripts/sma.py
/usr/libexec/spdk/scripts/spdk-gpt.py
/usr/libexec/spdk/scripts/spdkcli.py
/usr/libexec/spdk/scripts/spdx.sh
/usr/libexec/spdk/scripts/sync_dev_uevents.sh
/usr/libexec/spdk/scripts/vagrant
/usr/libexec/spdk/scripts/vagrant/README.md
/usr/libexec/spdk/scripts/vagrant/Vagrantfile
/usr/libexec/spdk/scripts/vagrant/autorun-spdk.conf
/usr/libexec/spdk/scripts/vagrant/create_nvme_img.sh
/usr/libexec/spdk/scripts/vagrant/create_vbox.sh
/usr/libexec/spdk/scripts/vagrant/local.conf
/usr/libexec/spdk/scripts/vagrant/run-autorun.sh
/usr/libexec/spdk/scripts/vagrant/update.sh
/usr/local/bin/fio
/usr/local/bin/fio/spdk_bdev
/usr/local/bin/fio/spdk_nvme
/usr/local/bin/iscsi_tgt
/usr/local/bin/nvmf_tgt
/usr/local/bin/spdk_dd
/usr/local/bin/spdk_lspci
/usr/local/bin/spdk_nvme_discovery_aer
/usr/local/bin/spdk_nvme_identify
/usr/local/bin/spdk_nvme_perf
/usr/local/bin/spdk_tgt
/usr/local/bin/spdk_top
/usr/local/bin/spdk_trace
/usr/local/bin/spdk_trace_record
/usr/local/bin/vhost
/usr/local/include/spdk
/usr/local/include/spdk/accel.h
/usr/local/include/spdk/accel_module.h
/usr/local/include/spdk/assert.h
/usr/local/include/spdk/barrier.h
/usr/local/include/spdk/base64.h
/usr/local/include/spdk/bdev.h
/usr/local/include/spdk/bdev_module.h
/usr/local/include/spdk/bdev_zone.h
/usr/local/include/spdk/bit_array.h
/usr/local/include/spdk/bit_pool.h
/usr/local/include/spdk/blob.h
/usr/local/include/spdk/blob_bdev.h
/usr/local/include/spdk/blobfs.h
/usr/local/include/spdk/blobfs_bdev.h
/usr/local/include/spdk/conf.h
/usr/local/include/spdk/config.h
/usr/local/include/spdk/cpuset.h
/usr/local/include/spdk/crc16.h
/usr/local/include/spdk/crc32.h
/usr/local/include/spdk/crc64.h
/usr/local/include/spdk/dif.h
/usr/local/include/spdk/dma.h
/usr/local/include/spdk/endian.h
/usr/local/include/spdk/env.h
/usr/local/include/spdk/env_dpdk.h
/usr/local/include/spdk/event.h
/usr/local/include/spdk/fd.h
/usr/local/include/spdk/fd_group.h
/usr/local/include/spdk/file.h
/usr/local/include/spdk/ftl.h
/usr/local/include/spdk/gpt_spec.h
/usr/local/include/spdk/hexlify.h
/usr/local/include/spdk/histogram_data.h
/usr/local/include/spdk/idxd.h
/usr/local/include/spdk/idxd_spec.h
/usr/local/include/spdk/init.h
/usr/local/include/spdk/ioat.h
/usr/local/include/spdk/ioat_spec.h
/usr/local/include/spdk/iscsi_spec.h
/usr/local/include/spdk/json.h
/usr/local/include/spdk/jsonrpc.h
/usr/local/include/spdk/likely.h
/usr/local/include/spdk/log.h
/usr/local/include/spdk/lvol.h
/usr/local/include/spdk/memory.h
/usr/local/include/spdk/mmio.h
/usr/local/include/spdk/nbd.h
/usr/local/include/spdk/notify.h
/usr/local/include/spdk/nvme.h
/usr/local/include/spdk/nvme_intel.h
/usr/local/include/spdk/nvme_ocssd.h
/usr/local/include/spdk/nvme_ocssd_spec.h
/usr/local/include/spdk/nvme_spec.h
/usr/local/include/spdk/nvme_zns.h
/usr/local/include/spdk/nvmf.h
/usr/local/include/spdk/nvmf_cmd.h
/usr/local/include/spdk/nvmf_fc_spec.h
/usr/local/include/spdk/nvmf_spec.h
/usr/local/include/spdk/nvmf_transport.h
/usr/local/include/spdk/opal.h
/usr/local/include/spdk/opal_spec.h
/usr/local/include/spdk/pci_ids.h
/usr/local/include/spdk/pipe.h
/usr/local/include/spdk/queue.h
/usr/local/include/spdk/queue_extras.h
/usr/local/include/spdk/reduce.h
/usr/local/include/spdk/rpc.h
/usr/local/include/spdk/scheduler.h
/usr/local/include/spdk/scsi.h
/usr/local/include/spdk/scsi_spec.h
/usr/local/include/spdk/sock.h
/usr/local/include/spdk/stdinc.h
/usr/local/include/spdk/string.h
/usr/local/include/spdk/thread.h
/usr/local/include/spdk/trace.h
/usr/local/include/spdk/trace_parser.h
/usr/local/include/spdk/tree.h
/usr/local/include/spdk/ublk.h
/usr/local/include/spdk/util.h
/usr/local/include/spdk/uuid.h
/usr/local/include/spdk/version.h
/usr/local/include/spdk/vfio_user_pci.h
/usr/local/include/spdk/vfio_user_spec.h
/usr/local/include/spdk/vfu_target.h
/usr/local/include/spdk/vhost.h
/usr/local/include/spdk/vmd.h
/usr/local/include/spdk/xor.h
/usr/local/include/spdk/zipf.h
/usr/local/lib/libspdk.so
/usr/local/lib/libspdk_accel.so
/usr/local/lib/libspdk_accel_ioat.so
/usr/local/lib/libspdk_bdev.so
/usr/local/lib/libspdk_bdev_aio.so
/usr/local/lib/libspdk_bdev_delay.so
/usr/local/lib/libspdk_bdev_error.so
/usr/local/lib/libspdk_bdev_ftl.so
/usr/local/lib/libspdk_bdev_gpt.so
/usr/local/lib/libspdk_bdev_lvol.so
/usr/local/lib/libspdk_bdev_malloc.so
/usr/local/lib/libspdk_bdev_null.so
/usr/local/lib/libspdk_bdev_nvme.so
/usr/local/lib/libspdk_bdev_passthru.so
/usr/local/lib/libspdk_bdev_raid.so
/usr/local/lib/libspdk_bdev_split.so
/usr/local/lib/libspdk_bdev_virtio.so
/usr/local/lib/libspdk_bdev_zone_block.so
/usr/local/lib/libspdk_blob.so
/usr/local/lib/libspdk_blob_bdev.so
/usr/local/lib/libspdk_blobfs.so
/usr/local/lib/libspdk_blobfs_bdev.so
/usr/local/lib/libspdk_conf.so
/usr/local/lib/libspdk_dma.so
/usr/local/lib/libspdk_env_dpdk.so
/usr/local/lib/libspdk_env_dpdk_rpc.so
/usr/local/lib/libspdk_event.so
/usr/local/lib/libspdk_event_accel.so
/usr/local/lib/libspdk_event_bdev.so
/usr/local/lib/libspdk_event_iobuf.so
/usr/local/lib/libspdk_event_iscsi.so
/usr/local/lib/libspdk_event_nbd.so
/usr/local/lib/libspdk_event_nvmf.so
/usr/local/lib/libspdk_event_scheduler.so
/usr/local/lib/libspdk_event_scsi.so
/usr/local/lib/libspdk_event_sock.so
/usr/local/lib/libspdk_event_vhost_blk.so
/usr/local/lib/libspdk_event_vhost_scsi.so
/usr/local/lib/libspdk_event_vmd.so
/usr/local/lib/libspdk_ftl.so
/usr/local/lib/libspdk_init.so
/usr/local/lib/libspdk_ioat.so
/usr/local/lib/libspdk_iscsi.so
/usr/local/lib/libspdk_json.so
/usr/local/lib/libspdk_jsonrpc.so
/usr/local/lib/libspdk_log.so
/usr/local/lib/libspdk_lvol.so
/usr/local/lib/libspdk_nbd.so
/usr/local/lib/libspdk_notify.so
/usr/local/lib/libspdk_nvme.so
/usr/local/lib/libspdk_nvmf.so
/usr/local/lib/libspdk_rpc.so
/usr/local/lib/libspdk_scheduler_dpdk_governor.so
/usr/local/lib/libspdk_scheduler_dynamic.so
/usr/local/lib/libspdk_scheduler_gscheduler.so
/usr/local/lib/libspdk_scsi.so
/usr/local/lib/libspdk_sock.so
/usr/local/lib/libspdk_sock_posix.so
/usr/local/lib/libspdk_thread.so
/usr/local/lib/libspdk_trace.so
/usr/local/lib/libspdk_trace_parser.so
/usr/local/lib/libspdk_ut_mock.so
/usr/local/lib/libspdk_util.so
/usr/local/lib/libspdk_vfio_user.so
/usr/local/lib/libspdk_vhost.so
/usr/local/lib/libspdk_virtio.so
/usr/local/lib/libspdk_vmd.so

With some core apps available:

[root@test-hostname spdk]# LIST_LIBS=yes  ./rpmbuild/rpm-deps.sh spdk_tgt
/usr/local/bin/spdk_tgt
/usr/lib64/libaio.so.1:libaio-0.3.111-15.fc38.x86_64
/usr/lib64/libcrypto.so.3:openssl-libs-3.0.9-2.fc38.x86_64
/usr/lib64/libc.so.6:glibc-2.37-4.fc38.x86_64
/usr/lib64/libgcc_s.so.1:libgcc-13.2.1-1.fc38.x86_64
/usr/lib64/libm.so.6:glibc-2.37-4.fc38.x86_64
/usr/lib64/libnuma.so.1:numactl-libs-2.0.16-2.fc38.x86_64
/usr/lib64/libssl.so.3:openssl-libs-3.0.9-2.fc38.x86_64
/usr/lib64/libuuid.so.1:libuuid-2.38.1-4.fc38.x86_64
/usr/lib64/libz.so.1:zlib-1.2.13-3.fc38.x86_64
/usr/local/lib/dpdk/librte_bus_pci.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_cryptodev.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_dmadev.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_eal.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_ethdev.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_hash.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_kvargs.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_mbuf.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_mempool_ring.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_mempool.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_meter.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_net.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_pci.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_power.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_rcu.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_ring.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_telemetry.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_timer.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/dpdk/librte_vhost.so.23:spdk-dpdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_accel_ioat.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_accel.so.13.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_aio.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_delay.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_error.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_ftl.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_gpt.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_lvol.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_malloc.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_null.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_nvme.so.6.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_passthru.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_raid.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev.so.13.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_split.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_virtio.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_bdev_zone_block.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_blob_bdev.so.10.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_blobfs_bdev.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_blobfs.so.9.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_blob.so.10.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_conf.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_dma.so.3.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_env_dpdk_rpc.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_env_dpdk.so.12.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_accel.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_bdev.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_iobuf.so.2.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_iscsi.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_nbd.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_nvmf.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_scheduler.so.3.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_scsi.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event.so.12.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_sock.so.4.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_vhost_blk.so.2.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_vhost_scsi.so.2.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_event_vmd.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_ftl.so.7.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_init.so.4.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_ioat.so.6.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_iscsi.so.7.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_jsonrpc.so.5.1:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_json.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_log.so.6.1:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_lvol.so.9.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_nbd.so.6.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_notify.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_nvme.so.11.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_nvmf.so.16.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_rpc.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_scheduler_dpdk_governor.so.3.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_scheduler_dynamic.so.3.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_scheduler_gscheduler.so.3.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_scsi.so.7.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_sock_posix.so.5.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_sock.so.8.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_thread.so.8.1:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_trace.so.8.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_util.so.7.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_vfio_user.so.4.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_vhost.so.7.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_virtio.so.6.0:spdk-libs-v24.01-1.x86_64
/usr/local/lib/libspdk_vmd.so.5.0:spdk-libs-v24.01-1.x86_64
[root@test-hostname spdk]# setup.sh
cleanup      config       help         interactive  reset        status
[root@test-hostname spdk]# rpc.py
Display all 262 possibilities? (y or n)
accel_assign_opc                         bdev_lvol_decouple_parent                bdev_nvme_stop_mdns_discovery            blobfs_create                            iscsi_set_options                        rpc_get_methods
accel_crypto_key_create                  bdev_lvol_delete                         bdev_ocf_create                          blobfs_detect                            iscsi_start_portal_group                 save_config
accel_crypto_key_destroy                 bdev_lvol_delete_lvstore                 bdev_ocf_delete                          blobfs_mount                             iscsi_target_node_add_lun                save_subsystem_config
accel_crypto_keys_get                    bdev_lvol_get_lvols                      bdev_ocf_flush_start                     blobfs_set_cache_size                    iscsi_target_node_add_pg_ig_maps         scsi_get_devices
accel_get_module_info                    bdev_lvol_get_lvstores                   bdev_ocf_flush_status                    compressdev_scan_accel_module            iscsi_target_node_remove_pg_ig_maps      sock_impl_get_options
accel_get_opc_assignments                bdev_lvol_grow_lvstore                   bdev_ocf_get_bdevs                       dpdk_cryptodev_get_driver                iscsi_target_node_request_logout         sock_impl_set_options
accel_get_stats                          bdev_lvol_inflate                        bdev_ocf_get_stats                       dpdk_cryptodev_scan_accel_module         iscsi_target_node_set_auth               sock_set_default_impl
accel_set_driver                         bdev_lvol_rename                         bdev_ocf_reset_stats                     dpdk_cryptodev_set_driver                iscsi_target_node_set_redirect           spdk_get_version
accel_set_options                        bdev_lvol_rename_lvstore                 bdev_ocf_set_cache_mode                  dsa_scan_accel_module                    load_config                              spdk_kill_instance
bdev_aio_create                          bdev_lvol_resize                         bdev_ocf_set_seqcutoff                   env_dpdk_get_mem_stats                   load_subsystem_config                    thread_get_io_channels
bdev_aio_delete                          bdev_lvol_set_read_only                  bdev_opal_create                         framework_disable_cpumask_locks          log_clear_flag                           thread_get_pollers
bdev_aio_rescan                          bdev_lvol_snapshot                       bdev_opal_delete                         framework_enable_cpumask_locks           log_enable_timestamps                    thread_get_stats
bdev_compress_create                     bdev_malloc_create                       bdev_opal_get_info                       framework_get_config                     log_get_flags                            thread_set_cpumask
bdev_compress_delete                     bdev_malloc_delete                       bdev_opal_new_user                       framework_get_pci_devices                log_get_level                            trace_clear_tpoint_mask
bdev_compress_get_orphans                bdev_null_create                         bdev_opal_set_lock_state                 framework_get_reactors                   log_get_print_level                      trace_disable_tpoint_group
bdev_crypto_create                       bdev_null_delete                         bdev_passthru_create                     framework_get_scheduler                  log_set_flag                             trace_enable_tpoint_group
bdev_crypto_delete                       bdev_null_resize                         bdev_passthru_delete                     framework_get_subsystems                 log_set_level                            trace_get_info
bdev_daos_create                         bdev_nvme_add_error_injection            bdev_raid_create                         framework_monitor_context_switch         log_set_print_level                      trace_get_tpoint_group_mask
bdev_daos_delete                         bdev_nvme_apply_firmware                 bdev_raid_delete                         framework_set_scheduler                  mlx5_scan_accel_module                   trace_set_tpoint_mask
bdev_daos_resize                         bdev_nvme_attach_controller              bdev_raid_get_bdevs                      framework_start_init                     nbd_get_disks                            ublk_create_target
bdev_delay_create                        bdev_nvme_cuse_register                  bdev_raid_remove_base_bdev               framework_wait_init                      nbd_start_disk                           ublk_destroy_target
bdev_delay_delete                        bdev_nvme_cuse_unregister                bdev_rbd_create                          iaa_scan_accel_module                    nbd_stop_disk                            ublk_get_disks
bdev_delay_update_latency                bdev_nvme_detach_controller              bdev_rbd_delete                          ioat_scan_accel_module                   notify_get_notifications                 ublk_start_disk
bdev_enable_histogram                    bdev_nvme_disable_controller             bdev_rbd_get_clusters_info               iobuf_set_options                        notify_get_types                         ublk_stop_disk
bdev_error_create                        bdev_nvme_enable_controller              bdev_rbd_register_cluster                iscsi_auth_group_add_secret              nvmf_create_subsystem                    vfu_tgt_set_base_path
bdev_error_delete                        bdev_nvme_get_controller_health_info     bdev_rbd_resize                          iscsi_auth_group_remove_secret           nvmf_create_transport                    vfu_virtio_create_blk_endpoint
bdev_error_inject_error                  bdev_nvme_get_controllers                bdev_rbd_unregister_cluster              iscsi_create_auth_group                  nvmf_delete_subsystem                    vfu_virtio_create_scsi_endpoint
bdev_examine                             bdev_nvme_get_discovery_info             bdev_reset_iostat                        iscsi_create_initiator_group             nvmf_get_stats                           vfu_virtio_delete_endpoint
bdev_ftl_create                          bdev_nvme_get_io_paths                   bdev_set_options                         iscsi_create_portal_group                nvmf_get_subsystems                      vfu_virtio_scsi_add_target
bdev_ftl_delete                          bdev_nvme_get_mdns_discovery_info        bdev_set_qd_sampling_period              iscsi_create_target_node                 nvmf_get_transports                      vfu_virtio_scsi_remove_target
bdev_ftl_get_stats                       bdev_nvme_get_path_iostat                bdev_set_qos_limit                       iscsi_delete_auth_group                  nvmf_set_config                          vhost_controller_set_coalescing
bdev_ftl_load                            bdev_nvme_get_transport_statistics       bdev_split_create                        iscsi_delete_initiator_group             nvmf_set_crdt                            vhost_create_blk_controller
bdev_ftl_unload                          bdev_nvme_opal_init                      bdev_split_delete                        iscsi_delete_portal_group                nvmf_set_max_subsystems                  vhost_create_scsi_controller
bdev_ftl_unmap                           bdev_nvme_opal_revert                    bdev_uring_create                        iscsi_delete_target_node                 nvmf_subsystem_add_host                  vhost_delete_controller
bdev_get_bdevs                           bdev_nvme_remove_error_injection         bdev_uring_delete                        iscsi_get_auth_groups                    nvmf_subsystem_add_listener              vhost_get_controllers
bdev_get_histogram                       bdev_nvme_reset_controller               bdev_virtio_attach_controller            iscsi_get_connections                    nvmf_subsystem_add_ns                    vhost_scsi_controller_add_target
bdev_get_iostat                          bdev_nvme_send_cmd                       bdev_virtio_blk_set_hotplug              iscsi_get_initiator_groups               nvmf_subsystem_allow_any_host            vhost_scsi_controller_remove_target
bdev_iscsi_create                        bdev_nvme_set_hotplug                    bdev_virtio_detach_controller            iscsi_get_options                        nvmf_subsystem_get_controllers           virtio_blk_create_transport
bdev_iscsi_delete                        bdev_nvme_set_multipath_policy           bdev_virtio_scsi_get_devices             iscsi_get_portal_groups                  nvmf_subsystem_get_listeners             virtio_blk_get_transports
bdev_iscsi_set_options                   bdev_nvme_set_options                    bdev_wait_for_examine                    iscsi_get_target_nodes                   nvmf_subsystem_get_qpairs                vmd_enable
bdev_lvol_clone                          bdev_nvme_set_preferred_path             bdev_xnvme_create                        iscsi_initiator_group_add_initiators     nvmf_subsystem_listener_set_ana_state    vmd_remove_device
bdev_lvol_clone_bdev                     bdev_nvme_start_discovery                bdev_xnvme_delete                        iscsi_initiator_group_remove_initiators  nvmf_subsystem_remove_host               vmd_rescan
bdev_lvol_create                         bdev_nvme_start_mdns_discovery           bdev_zone_block_create                   iscsi_portal_group_set_auth              nvmf_subsystem_remove_listener
bdev_lvol_create_lvstore                 bdev_nvme_stop_discovery                 bdev_zone_block_delete                   iscsi_set_discovery_auth                 nvmf_subsystem_remove_ns
[root@test-hostname spdk]# nvmf_tgt --
--base-virtaddr            --env-context              --iova-mode                --limit-coredump           --mem-channels             --num-trace-entries        --rpc-socket               --tpoint-group
--config                   --help                     --json                     --logflag                  --mem-size                 --pci-allowed              --shm-id                   --version
--cpumask                  --huge-dir                 --json-ignore-init-errors  --main-core                --msg-mempool-size         --pci-blocked              --silence-noticelog        --vfio-vf-token
--disable-cpumask-locks    --huge-unlink              --lcores                   --max-delay                --no-pci                   --rpcs-allowed             --single-file-segments     --wait-for-rpc

# fio plugins are included as well (albeit their location is not best)
[root@test-hostname spdk]# echo /usr/local/bin/fio/spdk_@(nvme|bdev)
/usr/local/bin/fio/spdk_bdev /usr/local/bin/fio/spdk_nvme

The reason I mention rpmbuild here is because as a end-user, I would not consider src installation as my first choice. Even on the embedded side, this installation would be wrapped by some env which puts everything in the right place (yocto|bitbake), without relaying solely on make install (having robust targets makes everything way easier though). And distro-wise, the first try is also usually packaging. In my mind I actually consider src installation as a last resort - mainly dedicated for testing, debugging, hotfixing, etc. but definitely not for regular deployment.

That said, having a bit more robust make targets wouldn't of course do any harm (as long as their complexity doesn't render the entire build|Makefile* infra incomprehensible). :)

benlwalker commented 1 year ago

The source install is nice because I can install into some other prefix under $HOME and keep the base system clean. I don't really want to get into containers because it adds extra steps to my workflow. I'm basically a developer making a pseudo-customer-environment for quick testing in this scenario.

I'm going to try to use the rpmbuild and see where I get now. The first obvious thing I see is that it requires rpmbuild, so doesn't that make this approach RH/CentOS/Fedora specific? That happens to be fine for me at the moment but Debian-variants are common enough that we may need something there too.

a16bitsysop commented 1 year ago

I was looking at this for alpine linux, and found arch linux spdk packaging renames some of the python scripts and manually installs them https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=spdk

But I found several scripts in spdk need rpc.py to be called rpc.py

tomzawadzki commented 1 year ago

[Bug scrub] @benlwalker will be working on some of the previously mentioned enhancements, to make sure that make install covers the right bases. Some of the ideas are: moving some examples (bdevperf/perf/etc) to apps, explore moving DPDK libs from rpmbuild.

mikeBashStuff commented 1 year ago

Debian-variants are common enough that we may need something there too.

I believe there's a series of patches on gerrit which introduce .deb packaging for SPDK. :)

I'm going to try to use the rpmbuild and see where I get now. The first obvious thing I see is that it requires rpmbuild, so doesn't that make this approach RH/CentOS/Fedora specific?

To build these, yes, it does. But for installation, not necessarily. Container-based POC below:

# Under ubuntu2204
$ git clone https://github.com/spdk/spdk --recurse-submodules
$ cd docker/build_base
$ tar -czf spdk.tar.gz --exclude=spdk.tar.gz -C ../../ . # building from local repo
# Build .rpms. configure's cmdline can be adjusted directly inside Dockerfile (it could be parametrized as well)
$ docker build --build-arg PROXY=$http_proxy --force-rm -t spdk-rpms:latest --target base .
# Fetch the rpms to the host side
$ mkdir -p rpms
$ docker run --name rpms --rm   -v "$PWD/rpms:/rpms" spdk-rpms:latest  bash -c 'cp /tmp/*.rpm /rpms/'
# Deploy - --directory is your DESTDIR
# This of course assumes that ubuntu2204 can provide compatible env similar to fedora image that was used for building these packages. This is mainly relevant for base|system set of libraries (like libssl, libnuma, etc.) since this is what actual rpm packages|SPDK would depend on
$ for r in  rpms/*.rpm; do rpm2cpio "$r" | cpio -i --directory=/ --make-directories --no-absolute-filenames; done

All the above is HACKy of course, so the more streamlined way would be to simply install rpmbuild under your non-rhel distro - for debian|ubuntu this is quite easy since these do provide rpm-related tooling in their repos (some minor tweaks would be necessary for the full build to work, but it's doable). Having completely separate set of .deb packages would be best of course.

POC showing use of rpm infra under debian|ubuntu:

# It would go more or less like this
$ apt-get install -y rpm
$ ln -sf /bin/bash /bin/sh # ubuntu|debian tends to link sh to dash by default. rpmbuild creates scripts with a shebang pointing at /bin/sh but to make sure they all run smoothly, it should provide access to bash
$ git diff # some extra tweaks are needed on our side
diff --git a/rpmbuild/rpm.sh b/rpmbuild/rpm.sh
index 667422c77..4e3f246de 100755
--- a/rpmbuild/rpm.sh
+++ b/rpmbuild/rpm.sh
@@ -14,15 +14,6 @@ source /etc/os-release

 id_ok=no

-for id in $ID $ID_LIKE; do
-       [[ "$id" =~ ^(fedora|centos|rhel) ]] && id_ok=yes
-done
-
-if [[ "$id_ok" != "yes" ]]; then
-       printf '%s not supported\n' "$ID" >&2
-       exit 1
-fi
-
 get_config() {
        # Intercept part of the ./configure's cmdline we are interested in
        configure_opts=($(getopt -l "$1::" -o "" -- $configure 2> /dev/null)) || true
@@ -61,6 +52,7 @@ fedora_python_sys_path_workaround() {

        PYTHONPATH="$(python3 -c "import sys; print('%s' % ':'.join(sys.path)[1:])")"
        export PYTHONPATH
+       macros+=(-D "python3_sitelib $(python3 -c "import sysconfig; print(sysconfig.get_paths()['purelib'])")") # this macro won't be available under ubuntu|debian by default
 }

 get_version() {
diff --git a/rpmbuild/spdk.spec b/rpmbuild/spdk.spec
index bf9a1fb5d..f48bb423b 100644
--- a/rpmbuild/spdk.spec
+++ b/rpmbuild/spdk.spec
@@ -36,8 +36,6 @@ Requires: zlib
 Requires: %(echo "%{requirements_list}")
 %endif

-BuildRequires: python3-devel # this is not needed at all really
-
 %if %{build_requirements}
 BuildRequires: %(echo "%{build_requirements_list}")
 %endif
$ DEPS=no ./rpmbuild/rpm.sh --with-shared # e.g.
$ rpm -i --nodeps /root/rpmbuild/rpm/x86_64/spdk-* # --nodeps since some of the dependencies simply won't exist under debian|ubuntu (won't exist in the sense that the naming of these packages would simply not match).

Also, just to note, this whole packaging effort was done purely for convenience's sake it definitely does not mean to enforce any specific approach to how SPDK should be installed|shipped in general. :)