Are partitions like nbd0p1 supported?

[alkisg]

Hi, the following fails, is it a supported use case? server: create a vm disk file named e.g. debian-raw with a partition table and a single ext4 partition. client:

nbd-client server -N debian-raw /dev/nbd0
mount -o ro /dev/nbd0p1 /mnt

The /dev/nbd0p1 node is successfully created, but mount complains about invalid format, and in syslog I see:

[ 288.548727] nbd0: p1 [ 288.548747] nbd0: p1 size 41934848 extends beyond EOD, truncated

On the other hand, if I use losetup, it works:

losetup /dev/loop5 /dev/nbd0
partprobe /dev/loop5
mount -o loop,ro /dev/loop5p1 /mnt

A nice use of this would be to netboot VMs, for example with: nbdroot=server:vmdisk root=/dev/nbd0p1 ...where manually running losetup there would require custom code inside the initramfs.

[yoe]

Should be supported, yes. Are you sure the partition is not too big? Are you perhaps exporting a device with size overridden?

[alkisg]

No the partition is fine, I tried with many different VM disks and they all have the same issue and they all work fine if I just use losetup /dev/loop0 /dev/nbd0. I'm not using a size override in the configuration, it's just 2 lines, the exportname and readonly=true.

I tried with hexdump and I saw that reading from nbd0p1 always returns 0, while if I read it indirectly using the loop0 device as I mentioned above, it returns the correct values:

root@pc:# hexdump -C -n 1024 /dev/nbd0p1
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400

root@pc:# hexdump -C -n 32 /dev/loop0p1
00000000  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|
00000010  00 00 00 00 00 f8 00 00  3f 00 ff 00 00 08 00 00  |........?.......|

[yoe]

That's... weird. What versions of everything are you using?

[alkisg]

Currently testing with the following on both the server and the client; I can test with anything else, if needed:

Linux alkis 4.15.0-38-generic #41-Ubuntu SMP Wed Oct 10 10:59:38 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ii  nbd-client     1:3.16.2-1ubuntu0.1 amd64        Network Block Device protocol - client
ii  nbd-server     1:3.16.2-1ubuntu0.1 amd64        Network Block Device protocol - server

[alkisg]

It works fine if I use a Debian Stretch VM as a client (while still using the Ubuntu 18.04 above as the nbd server), with the following versions:

Linux stretch-mate 4.9.0-7-686-pae #1 SMP Debian 4.9.110-1 (2018-07-05) i686 GNU/Linux
ii  nbd-client     1:3.15.2-3   i386         Network Block Device protocol - client

I'll do a few more tests on where it works and where it's broken, and report back. Thanks!

[alkisg]

So, on that Debian Stretch VM, if I upgrade the kernel from 4.9 to:

Linux stretch-mate 4.18.0-0.bpo.1-686-pae #1 SMP Debian 4.18.6-1~bpo9+1 (2018-09-13) i686 GNU/Linux

...it starts having the issue; i.e. it appears to be a regression in recent kernels.

[yoe]

@josefbacik any insight on that?

[mxork]

I've noticed this as well on recent (4.18.x, 4.19.x) Arch, but only with GPT. DOS part tables seem to consistently be found.

[yoe]

This was probably a result of e544541b0765c341174613b416d4b074fa7571c2 in the kernel (which was added to 4.10). I've now updated nbd-client (commit 128fd556286ff5d53c5f2b16c4ae5746b5268a64) so that the default block size becomes 512 bytes rather than 1024; that should hopefully fix the issue.

[alkisg]

Confirming that just passing -b 512 to nbd-client works around the issue, in all affected versions.

# Work around https://github.com/NetworkBlockDevice/nbd/issues/87
sed 's/systemd-mark/& -b 512/' -i /scripts/local-top/nbd

[alkisg]

@yoe, the initramfs /scripts/local-top/nbd script still has an issue, can I propose a workaround here?

When I set root=/dev/nbd0p1, I'm getting:

ALERT! /dev/nbd0p1 does not exist. Dropping to a shell!

To work around it, I patched the nbd script to call partprobe, maybe you could include that upstream?

sed 's/systemd-mark$/& -b 512; partprobe $nbdrootdev \&/' -i /scripts/local-top/nbd

[alkisg]

Partprobe isn't available in Ubuntu initramfs-tools, while blockdev is in both Debian and Ubuntu, so this one's better:

sed 's/systemd-mark$/& -b 512; blockdev --rereadpt $nbdrootdev/' -i /scripts/local-top/nbd