diskspd-for-linux

Summary

A disk io load-generator and benchmarking tool for Linux, based on the Windows tool diskspd.

Features

• Easy to read output showing IOPS, throughput (MB/s), average latency (-L) and standard deviations (-L -D)
• Multiple threads doing work on the same file or different files, various access patterns available (-t -F -s -T)
• CPU affinity - specify a set of CPU's to bind threads to (-a -n)
• Overlapped (async) io with choice of libaio (linux kernel aio) or posix aio (userspace threads) (-x)

Getting Started

Dependencies and System Requirements

• make
• gcc 5.4 or newer
• libaio-dev

Building and installing

Build

    make

Install (to /usr/local/bin)

    make install

Debug build (enables debugging output with d_printf() function):

    make DEBUG=1

You may need to make clean first if you already built diskspd without debugging turned on

Static linking (enables static libraries for binary portability):

    make STATIC=1

Usage

    diskspd [OPTIONS...] FILE [FILE...]

At least one file (or disk) must be specified, along with a number of optional arguments. Don't leave a space between the option and its argument; some arguments are optional and argp doesn't support spaces in this case. To have diskspd create files of a specified size, use -c followed by the file size.

To see the full list of options, use:

    diskspd --help

Examples and tips

By default, diskspd creates a single thread for each file or disk specified and does sequential IO operations starting at offset 0, keeping at most 2 operations in flight at a time per-thread. These operations are offset from each other by the stride (which defaults to the specified block or IO size, which itself defaults to 64K).

The most basic usage of the tool is to simply use -c to create a file to do IO on, and run with the defaults options enabled. This example creates a 1MiB file filled with a repeating pattern and reads from it sequentially until the test is done. The Linux kernel's asynchronous io interface is used to keep at most two operations in flight. A 5 second warmup is followed by 10 seconds of recording results.

    diskspd -c1M testfile

The following example writes directly (without buffering) to the partition /dev/sdc1 without touching the first 1GiB of the disk, using 1MiB blocks. Warm up is only 2 seconds, and results are recorded for 60 seconds. Output latency information and standard deviation for IOPS are displayed. 8 threads are used in total each keeping at most 4 operations in flight at a time, for a total of up to 32 in flight operations at a time.

    diskspd -b1M -B1G -w100 -Sh -W2 -d60 -L -D -t8 -o4 /dev/sdc1

Useful options:

• -F and -t set the total number of threads to use. Mutually exclusive with -t

• -d modifies the duration of the test (default is 10 seconds)

• -Sh opens files with O_DIRECT and O_SYNC flags, ensuring no caching is done in the OS. Note that all I/O must be aligned to the underlying device's physical sector size for this option to work.

• -t modifies how many threads do work on each file (default is 1)

• -W modifies the warmup time (default is 5 seconds)

• -z uses C++'s random_device to seed the offsets generated by -r (default is a seed of 0)

• -Zr uses separate read and write I/O buffers, and fill them with random data (default is ascending bytes)

More detailed information about diskspd's usage can be found in the wiki on GitHub.

Planned features

• Cooldown time (-C), same as warmup time but at the end of the duration
• fadvise flags to hint at file access patterns (analogous to Windows diskspd's -f<rst>)
• -h alternative for printing help
• I/O burst size (-i) and pause (-j). Issue bursts of IO with some milliseconds of pause in between them
• Use large pages for buffers (-l)
• JSON/XML input (-[XJ] FILE) and output (-R[jx]) (to file or just stdout)
  • specify most options at a per-target level using a JSON or XML file
  • output results to JSON or XML, possibly including more data (e.g. latency histogram data)
• Unit tests
• Validation/verification mode. Currently diskspd for linux errors out if bytes read/written aren't equal to block size (this only happens when there are alignment issues and such).

Known issues

• File setup quite slow for the first run
  • currently mitigated by simply using existing files if they're of the right size or larger
  • parallelizing file setup would improve this
• Some combinations of inputs may cause undefined behaviour
  • for the most part everything is checked for validity and conflicting options error out or are ignored, but it's possible not every combination is covered
• Possible bugs with Iops std-dev calculation - it seems to match diskspd for Windows behaviour however. This could be an issue with how the IoBucketizer is used, or something else.

Terms of Use

By downloading and running this project, you agree to the license terms of the third party application software, Microsoft products, and components to be installed.

The third party software and products are provided to you by third parties. You are responsible for reading and accepting the relevant license terms for all software that will be installed. Microsoft grants you no rights to third party software.

License

Copyright (c) 2018 Microsoft Corporation. All rights reserved.

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.