search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.7k stars 1.76k forks source link

Ubuntu 20.04 beta encrypted files disappear #10221

Closed mibolin closed 4 years ago

mibolin commented 4 years ago

System information

Type Version/Name
Distribution Name Ubuntu
Distribution Version 20.04
Linux Kernel 5.4.0-21-generic
Architecture x86_64
ZFS Version 0.8.3-1ubuntu8
SPL Version 0.8.3-1ubuntu8

Describe the problem you're observing

For benchmarking I create encrypted datasets and write a file with dd on it. After shutdown -h now and restarting the computer I have to manually import the pool and load the encryption keys. But, my benchmark file is not there anymore. It happened to me first on 2020/04/13 but I could reproduce it today (had no time before).

Describe how to reproduce the problem

Some comments about my setup. It's my old AM2 fileserver with an Athlon X2 cpu and ECC DDR2 memory. Before upgrading to a new ZFS mirror on SATA drives I connected 2xWD Elements 10TB external harddisks to the USB3 PCIe card. My / is on EXT4 on an old ssd.

zpool create -f daten2020 -m /mnt/daten2020 -o ashift=12 mirror /dev/sdg /dev/sdh
zfs set compression=lz4 daten2020
dd if=/dev/random of=/root/20200416_zfs_key.dd bs=32 count=1
zfs create -o encryption=aes-256-ccm -o keyformat=raw -o keylocation=file:///root/20200416_zfs_key.dd daten2020/enc256ccm_2
zfs create -o encryption=aes-256-gcm -o keyformat=raw -o keylocation=file:///root/20200416_zfs_key.dd daten2020/enc256gcm_2
root@fileserver:/home/michael# dd if=/dev/urandom of=/mnt/daten2020/enc256ccm_2/speed_daten2020_rand_bs4k_count10k.dd bs=4k count=10000 conv=fsync
10000+0 records in
10000+0 records out
40960000 bytes (41 MB, 39 MiB) copied, 1.21541 s, 33.7 MB/s
root@fileserver:/home/michael# dd if=/dev/urandom of=/mnt/daten2020/enc256gcm_2/speed_daten2020_rand_bs4k_count10k.dd bs=4k count=10000 conv=fsync
10000+0 records in
10000+0 records out
40960000 bytes (41 MB, 39 MiB) copied, 2.52985 s, 16.2 MB/s
root@fileserver:/home/michael# ll /mnt/daten2020/enc256gcm_2
total 40062
drwxr-xr-x 2 root root        3 Apr 16 21:15 ./
drwxrwxrwx 6 root root       10 Apr 16 21:14 ../
-rw-r--r-- 1 root root 40960000 Apr 16 21:15 speed_daten2020_rand_bs4k_count10k.dd
root@fileserver:/home/michael# ll /mnt/daten2020/enc256ccm_2
total 40062
drwxr-xr-x 2 root root        3 Apr 16 21:15 ./
drwxrwxrwx 6 root root       10 Apr 16 21:14 ../
-rw-r--r-- 1 root root 40960000 Apr 16 21:15 speed_daten2020_rand_bs4k_count10k.dd

So, the benchmark files were there. Then I shutdown -h now and restarted this morning.

root@fileserver:/home/michael# zpool import daten2020
root@fileserver:/home/michael# ll /mnt/daten2020/enc256ccm_2/
total 9
drwxr-xr-x 2 root root  2 Apr 16 21:13 ./
drwxrwxrwx 6 root root 10 Apr 16 21:14 ../
root@fileserver:/home/michael# ll /mnt/daten2020/enc256gcm_2/
total 9
drwxr-xr-x 2 root root  2 Apr 16 21:14 ./
drwxrwxrwx 6 root root 10 Apr 16 21:14 ../
root@fileserver:/home/michael# zfs load-key -a
4 / 4 key(s) successfully loaded
root@fileserver:/home/michael# ll /mnt/daten2020/enc256gcm_2/
total 9
drwxr-xr-x 2 root root  2 Apr 16 21:14 ./
drwxrwxrwx 6 root root 10 Apr 16 21:14 ../
root@fileserver:/home/michael# ll /mnt/daten2020/enc256ccm_2/
total 9
drwxr-xr-x 2 root root  2 Apr 16 21:13 ./
drwxrwxrwx 6 root root 10 Apr 16 21:14 ../
root@fileserver:/home/michael# zfs get -p encryption,keystatus,keylocation
NAME                   PROPERTY     VALUE                             SOURCE
daten                  encryption   off                               default
daten                  keystatus    -                                 -
daten                  keylocation  none                              default
daten@backup20190203   encryption   off                               default
daten@backup20190203   keystatus    -                                 -
daten@backup20190203   keylocation  -                                 -
daten@backup20190205   encryption   off                               default
daten@backup20190205   keystatus    -                                 -
daten@backup20190205   keylocation  -                                 -
daten@backup20200405   encryption   off                               default
daten@backup20200405   keystatus    -                                 -
daten@backup20200405   keylocation  -                                 -
daten2020              encryption   off                               default
daten2020              keystatus    -                                 -
daten2020              keylocation  none                              default
daten2020/enc256ccm    encryption   aes-256-ccm                       -
daten2020/enc256ccm    keystatus    available                         -
daten2020/enc256ccm    keylocation  file:///root/20200412_zfs_key.dd  local
daten2020/enc256ccm_2  encryption   aes-256-ccm                       -
daten2020/enc256ccm_2  keystatus    available                         -
daten2020/enc256ccm_2  keylocation  file:///root/20200416_zfs_key.dd  local
daten2020/enc256gcm    encryption   aes-256-gcm                       -
daten2020/enc256gcm    keystatus    available                         -
daten2020/enc256gcm    keylocation  file:///root/20200412_zfs_key.dd  local
daten2020/enc256gcm_2  encryption   aes-256-gcm                       -
daten2020/enc256gcm_2  keystatus    available                         -
daten2020/enc256gcm_2  keylocation  file:///root/20200416_zfs_key.dd  local
root@fileserver:/home/michael#

So, the benchmark files in /mnt/daten2020/enc256ccm_2/ and /mnt/daten2020/enc256gcm_2/ are gone. Why?

Include any warning/errors/backtraces from the system logs

I haven't seen warnings/errors. Please ask for more details.

mibolin commented 4 years ago

As Ubuntu 20.04 gets updates I re-tested again. The problem is still reproducible. The written files are not there after a reboot. If my testing is wrong - please tell me.

Currently used versions: `root@fileserver:/home/michael# uname -a Linux fileserver 5.4.0-25-generic #29-Ubuntu SMP Fri Apr 17 15:06:57 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

root@fileserver:/home/michael# modinfo zfs | grep -iw version version: 0.8.3-1ubuntu11

root@fileserver:/home/michael# modinfo spl | grep -iw version version: 0.8.3-1ubuntu11 `

InsanePrawn commented 4 years ago

I suspect your encrypted datasets are just not mounted. Check the output of df -h /mnt/daten2020/enc256ccm_2, mountpoint /mnt/daten2020/enc256ccm_2 or even a simple mount | grep /mnt/daten2020/enc256ccm_2.

If the post-reboot terminal session you posted is complete, that's unsurprising; after loading the keys with zfs load-key -a, you never mount anything. Try a zfs mount -a after loading the keys?

mibolin commented 4 years ago

Thank you very much. I didn't know that I have to mount the datasets after zfs load-key -a. I thought this is done automatically. With zfs mount -a the "old" files are still there.

Sorry for the noise. This issue was not a ZFS issue. Closed.