nbdkit is an NBD server. NBD — Network Block Device — is a protocol for accessing Block Devices (hard disks and disk-like things) over a Network.
The key features of nbdkit are:
For documentation, see the docs directory.
For plugins and filters, see the plugins and
filters directories. Example plugins can be found in
plugins/example*
.
There are many NBD clients, but the nbdkit project has a companion client library called https://gitlab.com/nbdkit/libnbd
https://gitlab.com/nbdkit/nbdkit
This software is Copyright Red Hat and licensed under a BSD license. See LICENSE for details.
Recommended for TLS (authentication and encryption) support, but it is possible to build without it:
For Windows support see the Windows section below.
For macOS support see the macOS section below.
To build the man pages, you will optionally need to install:
For SELinux socket labelling support:
For the gzip filter:
For the xz filter:
For the memory plugin with allocator=zstd:
For the curl (HTTP/FTP) plugin:
For the ssh plugin:
For the iso plugin:
For the floppy plugin:
For the libvirt plugin:
For the libguestfs plugin, and to run parts of the test suite:
For the ext2 filter:
For the linuxdisk plugin:
For the nbd plugin, to get URI and TLS support, and also to run parts of the test suite:
For the bittorrent plugin:
For the blkio plugin:
For the containerized data importer (CDI) plugin:
For the Perl and example4 plugins:
For the Python plugin:
For the OCaml plugin:
For the Tcl plugin:
For the Lua plugin:
For the Rust plugin:
To be able to write plugins in golang:
For bash tab completion:
To test for memory leaks (make check-valgrind
):
For non-essential enhancements to the test suite:
To build from tarball: To build from git:
autoreconf -i
./configure ./configure
make make
make check make check
On FreeBSD and OpenBSD which do not have GNU make by default you must
use gmake
instead of make
.
To run nbdkit from the source directory, use the top level ./nbdkit wrapper. It will run nbdkit and plugins from the locally compiled directory:
$ ./nbdkit example1 -f -v
./server/nbdkit ./plugins/example1/.libs/nbdkit-example1-plugin.so -f -v
[etc]
Optionally run this command as root to install everything:
make install
Since nbdkit >= 1.16, only Python >= 3.6 is supported.
By default nbdkit uses the Python version of the Python interpreter called “python” on the current $PATH. If you have parallel versions of Python installed then you can choose a different version by setting the PYTHON variable when configuring. For example:
./configure PYTHON=/usr/bin/python3.8
Most recent (less than, say, 8 years old) versions of the OCaml compiler should work. By default the configure script should detect the bytecode and native code compilers and enable either one or both.
To enable OCaml warnings and turn them into hard errors (developers only):
./configure OCAMLOPTFLAGS="-warn-error +A-3"
To run the test suite:
make check
If there is a failure, look at the corresponding tests/*.log
file
for debug information.
A few tests require root privileges, and are skipped by default. To run them you must do:
make check-root
If you have the proprietary VDDK library, you can test nbdkit-vddk-plugin against the library like this:
make check-vddk vddkdir=vmware-vix-disklib-distrib
To run benchmarks:
make bench
Tarballs are available from: http://libguestfs.org/download/nbdkit
If you are packaging nbdkit, use:
./configure --with-extra='...'
providing extra information about the distribution, and/or distro-specific versions. It helps us with troubleshooting bug reports. (Also, talk to us!)
Install the valgrind program and development headers.
Use:
./configure --enable-gcc-warnings --enable-valgrind
When testing use:
make check
make check-valgrind
For development ideas, see the TODO file.
The upstream git repository is: https://gitlab.com/nbdkit/nbdkit
Please send patches to the libguestfs mailing list: https://www.redhat.com/mailman/listinfo/libguestfs
For further information, see: https://libguestfs.org/ https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md
You can compile nbdkit with clang and ASAN with:
./configure CC=clang CXX=clang++ \
CFLAGS="-O0 -g -fsanitize=address -fno-omit-frame-pointer" \
--disable-linker-script \
--disable-golang
make clean
make
ASAN_OPTIONS="allocator_may_return_null=1 detect_leaks=false" make check
./configure CFLAGS="--coverage -g" LDFLAGS="--coverage -g"
make clean
make
make check
lcov -c -d . -o gcov.info
genhtml -o coverage gcov.info
Open your browser and examine the coverage/
directory. At the time
of writing (2020-04) test coverage of the server is reasonable, but
things are much worse for certain plugins and filters.
nbdkit works on macOS and is mostly feature complete. Some tests may fail or be skipped.
We have tested macOS 12 (Monterey) using MacPorts. Other versions of macOS and Homebrew may work too.
We installed the following MacPorts packages:
We configured nbdkit using:
./configure --enable-gcc-warnings --disable-perl \
CFLAGS="-g -O2 -I/opt/local/include" \
LDFLAGS="-L/opt/local/lib"
Experimentally, the server can be compiled on Windows or cross-compiled from Linux using mingw-w64. Only a small subset of features are available. To find out what is missing read the TODO "Windows port".
Note that GCC or Clang is required even if you are compiling on Windows because of some C language extensions we use. MSVC will not work.
For the rest of this section we talk about cross-compiling for Windows using Linux and mingw-w64. At a minimum you will need:
You may want to patch Wine with this patch which adds support for AF_UNIX sockets. It is needed to get most of the test suite to work, but if you don't care about the -U option then it is not needed. https://www.winehq.org/pipermail/wine-devel/2021-May/187049.html
Other mingw-w64 libraries may be installed which will add functionality (see full list of requirements above), but you may end up hitting areas we have not compiled or tested before.
To cross compile do:
mingw64-configure --disable-ocaml --disable-perl --disable-vddk
mingw64-make
You can test if the server is working by doing:
./nbdkit.exe --dump-config
(This usually runs wine automatically. If not, you may need to prefix the command "wine nbdkit.exe ...")
To see which plugins and filters were compiled:
find plugins filters -name '*.dll'
You can run them under Wine without installing using eg:
./nbdkit.exe -f -v memory 1G
You can run the test suite in the usual way:
mingw64-make check
To test the binary on a real Windows system you will need to copy
server/.libs/nbdkit.exe, any plugins and filters you need
(plugins/*/.libs/*.dll
and filters/*/.libs/*.dll
), and many mingw
DLLs (libdl.dll
, libgnutls.dll
, libwinpthread.dll
, etc.).
Notes:
libtool creates files called nbdkit.exe in the top level directory
and in server. These are the stripped binaries. The
"real" binaries are located in hidden .libs/
subdirectories.
The top level .libs/nbdkit.exe
is the wrapper used to run nbdkit
from the build directory, not the server binary. You need
server/.libs/nbdkit.exe
instead.
To find the list of mingw DLLs I ran nbdkit.exe iteratively until Windows stopped complaining about missing libraries. You could also use a tool like Dependency Walker.
If you copy everything into a single directory on the Windows server then you should be able to run nbdkit normally, eg:
nbdkit.exe -f -v nbdkit-memory-plugin.dll 1G
You will probably need to open port 10809 on the Windows Firewall.