openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.68k stars 1.76k forks source link

File truncate operation does not work with dedup on and fast dedup enabled #16718

Closed serjponomarev closed 3 weeks ago

serjponomarev commented 3 weeks ago

I decided to test file deletion as described in issue 16697 with FDT enabled but using file truncation as described in issue 16708 and found that with deduplication enabled with the FDT feature, file truncation does not work.

To check this, simply create a file: truncate -s 1T /zpool/test.io Check the file size: du -h --apparent-size /zpool/test.io 1.0T /zpool/test.io

Then truncate the file, for example: truncate -s 1G /zpool/test.io Check the file size: du -h --apparent-size /zpool/test.io 1.0T /zpool/test.io The file size will not change.

If you reduce the file size to 0, the truncation operation will work.

If you create a pool with deduplication enabled, but without the FDT feature, this problem does not occur.

amotin commented 3 weeks ago

I have difficulties to understand how it might be related to FDT, but not to traditional dedup, or how it can be related to dedup at all. I am not sure what exactly du -h --apparent-size shows. What does ls -l show you? If you want to exclude a transient effects, I'd make sure to add zpool sync calls in strategic places before you measure any disk usage.

serjponomarev commented 3 weeks ago

I have difficulties to understand how it might be related to FDT, but not to traditional dedup, or how it can be related to dedup at all. I am not sure what exactly du -h --apparent-size shows. What does ls -l show you? If you want to exclude a transient effects, I'd make sure to add zpool sync calls in strategic places before you measure any disk usage.

>> I am not sure what exactly du -h --apparent-size Shows the file size. Not the actual used size, but the reserved size, as it is shown for sparse files.

What I tried before posting this issue:

  1. zpool sync
  2. reduced txg timeout to 1 second
  3. made a snapshot to flush the data

So, I also don't understand why truncate and file resizing doesn't happen with deduplication and FDT enabled, but nevertheless, this issue is reproducible. Check it out, it will take a couple of minutes.

allanjude commented 3 weeks ago

Can you inspect the file before and after the truncation to the smaller size, with: zdb -dddddd -bbbbbb $pool/$dataset $inode-number

As you mentioned, you might need to zpool sync before the zdb run for it to show up

Does the DMU see the resize happen?

serjponomarev commented 3 weeks ago

The issue turned out to be with ZFS, which another engineer had built from source. After I rebuilt ZFS from the source, the problem disappeared, and I was unable to reproduce it. Why that particular build behaved this way remains a mystery.

I apologize for any confusion caused.