Internal error: Referenced LV pvmove0 not listed

swapdisk commented 5 months ago

I've run into what looks like a bug with the pvmove utility. I have managed to narrow it down the following easy reproducer. The pvmove fails with "Internal error" if given a PE range that happens to include LE's from more that one LV. For example...

[root@viable-martin ~]# vgcreate testvg /dev/nvme1n1p1
  Physical volume "/dev/nvme1n1p1" successfully created.
  Not creating system devices file due to existing VGs.
  Volume group "testvg" successfully created
[root@viable-martin ~]# lvcreate -l 256 -n lv1 testvg
  Logical volume "lv1" created.
[root@viable-martin ~]# lvcreate -l 256 -n lv2 testvg
  Logical volume "lv2" created.
[root@viable-martin ~]# pvdisplay -m /dev/nvme1n1p1
  --- Physical volume ---
  PV Name               /dev/nvme1n1p1
  VG Name               testvg
  PV Size               <4.00 GiB / not usable 3.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              1023
  Free PE               511
  Allocated PE          512
  PV UUID               obcJmt-rSok-IZ7M-Pvnu-udIz-CuOl-LotH2j

  --- Physical Segments ---
  Physical extent 0 to 255:
    Logical volume  /dev/testvg/lv1
    Logical extents 0 to 255
  Physical extent 256 to 511:
    Logical volume  /dev/testvg/lv2
    Logical extents 0 to 255
  Physical extent 512 to 1022:
    FREE

[root@viable-martin ~]# pvmove --alloc anywhere /dev/nvme1n1p1:250-260
  Internal error: Referenced LV pvmove0 not listed in VG testvg.
[root@viable-martin ~]# echo $?
5

The given PE range 250-260 includes some LE's from lv1 and some from lv2 and that's the condition that reproduces the issue.

Observed behavior

The pvmove fails with "Internal error: Referenced LV pvmove0 not listed in VG" and exit status 5.

Expected behavior

With plenty of free PE's available, the pvmove command should happily move the requested range of PE's.

Manual workaround

The expected behavior can be achieved by running pvmove more than once giving PE ranges that include LE's from only one LV at a time, for example:

[root@viable-martin ~]# pvmove --alloc anywhere /dev/nvme1n1p1:250-255
  /dev/nvme1n1p1: Moved: 100.00%
[root@viable-martin ~]# pvmove --alloc anywhere /dev/nvme1n1p1:256-260
  /dev/nvme1n1p1: Moved: 100.00%
[root@viable-martin ~]# pvdisplay -m /dev/nvme1n1p1
  --- Physical volume ---
  PV Name               /dev/nvme1n1p1
  VG Name               testvg
  PV Size               <4.00 GiB / not usable 3.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              1023
  Free PE               511
  Allocated PE          512
  PV UUID               obcJmt-rSok-IZ7M-Pvnu-udIz-CuOl-LotH2j

  --- Physical Segments ---
  Physical extent 0 to 249:
    Logical volume  /dev/testvg/lv1
    Logical extents 0 to 249
  Physical extent 250 to 260:
    FREE
  Physical extent 261 to 511:
    Logical volume  /dev/testvg/lv2
    Logical extents 5 to 255
  Physical extent 512 to 517:
    Logical volume  /dev/testvg/lv1
    Logical extents 250 to 255
  Physical extent 518 to 522:
    Logical volume  /dev/testvg/lv2
    Logical extents 0 to 4
  Physical extent 523 to 1022:
    FREE

More info

[root@viable-martin ~]# uname -r
5.14.0-427.22.1.el9_4.x86_64
[root@viable-martin ~]# rpm -qf $(which pvmove)
lvm2-2.03.23-2.el9.x86_64

Kindly let me know if there's any further information you need.

swapdisk commented 4 months ago

Here's another example reproducer. This one is using latest Fedora 40...

root@fedora:~# uname -r
6.9.7-200.fc40.x86_64
root@fedora:~# rpm -qf $(which pvmove)
lvm2-2.03.23-1.fc40.x86_64
root@fedora:~# vgcreate testvg /dev/vdb5
  Physical volume "/dev/vdb5" successfully created.
  Creating devices file /etc/lvm/devices/system.devices
  Volume group "testvg" successfully created
root@fedora:~# lvcreate -l 6 -n lv1 testvg
  Logical volume "lv1" created.
root@fedora:~# lvcreate -l 6 -n lv2 testvg
  Logical volume "lv2" created.
root@fedora:~# pvdisplay -m /dev/vdb5
  --- Physical volume ---
  PV Name               /dev/vdb5
  VG Name               testvg
  PV Size               100.00 MiB / not usable 4.00 MiB
  Allocatable           yes 
  PE Size               4.00 MiB
  Total PE              24
  Free PE               12
  Allocated PE          12
  PV UUID               PWI6f9-spby-F3Qj-mxXH-1Z7I-L4Yl-f2DXIB

  --- Physical Segments ---
  Physical extent 0 to 5:
    Logical volume  /dev/testvg/lv1
    Logical extents 0 to 5
  Physical extent 6 to 11:
    Logical volume  /dev/testvg/lv2
    Logical extents 0 to 5
  Physical extent 12 to 23:
    FREE

root@fedora:~# pvmove --alloc anywhere /dev/vda5:4-7
  Cannot use /dev/vda5: device not found
  Run `pvmove --help' for more information.
root@fedora:~# pvmove --alloc anywhere /dev/vdb5:4-7
  Internal error: Referenced LV pvmove0 not listed in VG testvg.
root@fedora:~# echo $?

zkabelac commented 3 months ago

Fixed with upstream commit https://github.com/lvmteam/lvm2/commit/07b6174f7cb3ac70df2791d54f654d6d5ca36169 Thanks for noticing.

lvmteam / lvm2