Linux ubuntu 5.4.0-24-generic #28-Ubuntu SMP Thu Apr 9 22:16:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Architecture
x86_64
ZFS Version
0.8.3-1ubuntu11
SPL Version
0.8.3-1ubuntu11
zfs --version
zfs-0.8.3-1ubuntu12 zfs-kmod-0.8.3-1ubuntu11
Describe the problem you're observing
I wanted to replicate the scenario described in https://jrs-s.net/2016/05/02/zfs-copies-equals-n/. My thinking is that although --copies=2 does not guarantee each copy will be on a separate disk, and neither that the metadata will be spread out, it should be possible to recover something even if a drive fails. I built the zfs pool using files, not loop-mounted devices.
I booted computer from Ubuntu 20.04 Daily 2020-04-16 installation iso, loaded onto USB disk.
When I issued zfs import to import my damaged pool, the command froze, all other zfs or zpool commands (except for trivial ones like --version) would also freeze, and system reboot also froze, requiring me to reboot using SysRq shortcut.
I tried again after reboot and also got the same result, that is handing zpool import and zfs commands hanging too. The dmesg messages were also similar.
When this happened for the first time, I had another pool mounted. It looked unafected (I could ls the mounted files just fine). I was afraid to touch it more, so I umounted it quickly.
Describe how to reproduce the problem
Create the pool
ubuntu@ubuntu:~$ for i in {1..2}; do dd if=/dev/zero of=/tmp/file$i bs=512M count=1 &> /dev/null; done
ubuntu@ubuntu:~$ sudo -i
root@ubuntu:~# zpool create test /tmp/file1 /tmp/file2
root@ubuntu:~# zpool status test
pool: test
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
test ONLINE 0 0 0
/tmp/file1 ONLINE 0 0 0
/tmp/file2 ONLINE 0 0 0
errors: No known data errors
Put two files on it
root@ubuntu:~# dd if=/dev/urandom bs=4M count=1 of=/test/test2
1+0 records in
1+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.030847 s, 136 MB/s
root@ubuntu:~# dd if=/dev/urandom bs=4M count=1 of=/test/test1
1+0 records in
1+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0301516 s, 139 MB/s
root@ubuntu:~# ls /test
test1 test2
Check I can export and reimport
root@ubuntu:~# zpool export test
root@ubuntu:~# zpool import -d /tmp test
First test, export the pool, and overwrite one of the backing files. Then reimport. Test passed.
root@ubuntu:~# zpool export test
root@ubuntu:~# dd if=/dev/zero of=/tmp/file2 bs=512M count=1 &> /dev/null
root@ubuntu:~# zpool import -d /tmp test
root@ubuntu:~# ls /test
test1 test2
root@ubuntu:~# zpool status
pool: test
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
test ONLINE 0 0 0
/tmp/file1 ONLINE 0 0 0
/tmp/file2 ONLINE 0 0 0
errors: No known data errors
Second test, export the pool and delete one of the files. Expected result is that I can no longer import. Test passed.
root@ubuntu:~# zpool export test
root@ubuntu:~# ls /test
ls: cannot access '/test': No such file or directory
root@ubuntu:~# rm /tmp/file2
root@ubuntu:~# zpool import -d /tmp test
cannot import 'test': no such pool or dataset
Destroy and re-create the pool from
a backup source.
root@ubuntu:~# zpool import -d /tmp
pool: test
id: 4376589746374121437
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://zfsonlinux.org/msg/ZFS-8000-3C
config:
test UNAVAIL insufficient replicas
/tmp/file1 ONLINE
/tmp/file2 UNAVAIL cannot open
Now, replace the missing file with a blank file. I now expect that this will import ok, and, if I am lucky, one or both files will be restored from undamaged copy. The zpool import hangs.
This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.
System information
zfs-kmod-0.8.3-1ubuntu11
Describe the problem you're observing
I wanted to replicate the scenario described in https://jrs-s.net/2016/05/02/zfs-copies-equals-n/. My thinking is that although --copies=2 does not guarantee each copy will be on a separate disk, and neither that the metadata will be spread out, it should be possible to recover something even if a drive fails. I built the zfs pool using files, not loop-mounted devices.
I booted computer from Ubuntu 20.04 Daily 2020-04-16 installation iso, loaded onto USB disk.
When I issued
zfs import
to import my damaged pool, the command froze, all otherzfs
orzpool
commands (except for trivial ones like--version
) would also freeze, and system reboot also froze, requiring me to reboot using SysRq shortcut.I tried again after reboot and also got the same result, that is handing
zpool import
and zfs commands hanging too. The dmesg messages were also similar.When this happened for the first time, I had another pool mounted. It looked unafected (I could ls the mounted files just fine). I was afraid to touch it more, so I
umount
ed it quickly.Describe how to reproduce the problem
Create the pool
Put two files on it
Check I can export and reimport
First test, export the pool, and overwrite one of the backing files. Then reimport. Test passed.
Second test, export the pool and delete one of the files. Expected result is that I can no longer import. Test passed.
Now, replace the missing file with a blank file. I now expect that this will import ok, and, if I am lucky, one or both files will be restored from undamaged copy. The
zpool import
hangs.Include any warning/errors/backtraces from the system logs
dmesg output
another dmesg, when I encountered issue first time
restarting zsysd would also hang, here's output when running under strace
zfs list under strace, also hangs