free space listed by "zfs list" increases over time

[ghost]

System information

Type	Version/Name
Distribution Name	Ubuntu
Distribution Version	16.04.3 LTS
Linux Kernel	4.11.0-14-generic
Architecture	x86_64
ZFS Version	0.7.0-86_g5df5d06
SPL Version	0.7.0-12_g9df9692

Describe the problem you're observing

The amount of free space listed by zfs list seems to keep increasing dramatically over time.

I have a 3x4TB raid5 array that appears as 7.19T in zpool list, which seems correct, but currently it says 1.38P free in zfs list. And two days ago it said 562T. The only thing that has changed is I've been steadily copying data into the pool. Nothing regarding the disk array or hardware has changed at all.

Describe how to reproduce the problem

Simply run zfs list as time progresses.

Include any warning/errors/backtraces from the system logs

The filesystem settings that I've changed from the defaults are encryption, compression and deduplication:

$ zfs get encryption,compress,dedup
NAME             PROPERTY     VALUE          SOURCE
storage          encryption   off            default
storage          compression  lz4            local
storage          dedup        off            default
storage/storage  encryption   aes-256-ccm    -
storage/storage  compression  lz4            local
storage/storage  dedup        on             local

Current zpool list:

$ zpool list
NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage  7.19T   230G  6.96T         -     1%     3%  1.13x  ONLINE  -

zfs list from two days ago:

$ zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
storage           171G   562T    24K  none

zfs list from today:

$ zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
storage           241G  1.38P    24K  none

[DeHackEd]

You are using dedup. This will cause the amount of "free" space to be faked since you're writing but not consuming more disk space. Of course, dedup is a whole other can of worms.

1.38P seems a bit much though for such an abysmal dedup ratio.

[ghost]

The dedup ratio is now lower, and the amount of free space is showing much higher:

$ zpool list
NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage  7.19T  2.53T  4.66T         -    11%    35%  1.09x  ONLINE  -
$ zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
storage          2.65T  7.95P    24K  none

[ghost]

Now it is showing well over 21PB and a dedup ratio under 1x...

$ zpool list
NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
storage  7.19T  5.29T  1.90T         -    27%    73%  0.21x  ONLINE  -
$ zfs list
NAME              USED  AVAIL  REFER  MOUNTPOINT
storage          5.68T  21.6P    24K  none

[DeHackEd]

A dedup ratio under 1.00x should be impossible. We definitely have a bug.

[GregorKopka]

Also: how can zfs AVAIL be higher than zpool FREE?

@behlendorf shouldn't this be a 0.7.x bug instead of a 0.8 milestone?

[behlendorf]

I'd love to get this fixed in the 0.7.x series if possible. But that's going to depend on us being able to reproduce this issue so it can be root caused. If someone can a simple reproducer that would be very helpful.

[behlendorf]

@bparker06 do you know if this is still an issue with the 0.8.0-rc1 or 0.7.11 tags?

[stale[bot]]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

[ghost]

I no longer use dedupe so I am unable to confirm, sorry.

[stale[bot]]

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

[GregorKopka]

I object.

[bghira]

I object.

if objecting, can you provide test data? no one else can or appears to be interested in doing so.