gaowayne commented 3 years ago

ISSUE:

if I send 4K io, it is OK. does not work with 256B and 512B.

[root@localhost wayne]# fio pmemblk256b.fio
pmemblk-write: (g=0): rw=randwrite, bs=(R) 256B-256B, (W) 256B-256B, (T) 256B-256B, ioengine=pmemblk, iodepth=1
...
pmemblk-read: (g=1): rw=randread, bs=(R) 256B-256B, (W) 256B-256B, (T) 256B-256B, ioengine=pmemblk, iodepth=1
...
fio-3.27
Starting 32 threads
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31639, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31642, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: pid=31644, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31649, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: pid=31643, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31640, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31647, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31650, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: pid=31652, err=22/file:ioengines.c:390, func=td_io_queue, error=Invalid argument
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.
fio: first direct IO errored. File system may not support direct IO, or iomem_align= is bad, or invalid block size. Try setting direct=0.

my fio is:

global]
bs=256b
ioengine=pmemblk
norandommap
time_based
runtime=3600
group_reporting
disable_lat=1
disable_slat=1
disable_clat=1
clat_percentiles=0
cpus_allowed_policy=split

# For the pmemblk engine:
#
#   IOs always complete immediately
#   IOs are always direct
#   Must use threads
#
iodepth=1
direct=1
thread
numjobs=16
#
# Unlink can be used to remove the files when done, but if you are
# using serial runs with stonewall, and you want the files to be created
# only once and unlinked only at the very end, then put the unlink=1
# in the last group.  This is the method demonstrated here.
#
# Note that if you have a read-only group and if the files will be
# newly created, then all of the data will read back as zero and the
# read will be optimized, yielding performance that is different from
# that of reading non-zero blocks (or unoptimized zero blocks).
#
unlink=0
#
# The pmemblk engine does IO to files in a DAX-mounted filesystem.
# The filesystem should be created on an NVDIMM (e.g /dev/pmem0)
# and then mounted with the '-o dax' option.  Note that the engine
# accesses the underlying NVDIMM directly, bypassing the kernel block
# layer, so the usual filesystem/disk performance monitoring tools such
# as iostat will not provide useful data.
#
# Here we specify a test file on each of two NVDIMMs.  The first
# number after the file name is the block size in bytes (4096 bytes
# in this example).  The second number is the size of the file to
# create in MiB (1 GiB in this example); note that the actual usable
# space available to fio will be less than this as libpmemblk requires
# some space for metadata.
#
# Currently, the minimum block size is 512 bytes and the minimum file
# size is about 17 MiB (these are libpmemblk requirements).
#
# While both files in this example have the same block size and file
# size, this is not required.
#
filename=/mnt/pmem0/fio-test,512,1024
#filename=/pmem1/fio-test,4096,1024

[pmemblk-write]
rw=randwrite
stonewall

[pmemblk-read]
rw=randread
stonewall
#
# We're done, so unlink the file:
#
unlink=1

Environment Information

PMDK package version(s):
OS(es) version(s):
ndctl version(s):
kernel version(s):
compiler, libraries, packaging and other related tools version(s):

Please provide a reproduction of the bug:

How often bug is revealed: (always, often, rare):

Actual behavior:

Expected behavior:

Details

Additional information about Priority and Help Requested:

Are you willing to submit a pull request with a proposed change? (Yes, No)

Requested priority: (Showstopper, High, Medium, Low)

lplewa commented 3 years ago

The minimum libpmemblk block size is 512, so 256 is incorrect(this is a number passed after the file path). FIO block size (bs argument) must by multiple of block size. In your file, you have 256 IO where the block size is 512, and this is why your IOs fail.