search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.44k stars 1.73k forks source link

Wrong `du`/`zfs list` output #10042

Open edo1 opened 4 years ago

edo1 commented 4 years ago

System information

Type Version/Name
Distribution Name Debian
Distribution Version Buster
Linux Kernel 5.4.8-1~bpo10+1
Architecture amd64
ZFS Version 0.8.3-1~bpo10+1
SPL Version 0.8.3-1~bpo10+1

Describe the problem you're observing

I found wrong allocated size displaying on my pool (displayed size is less than file size even on incompressible files).

Describe how to reproduce the problem

It seems that a combination of raidz/raidz2, non-default ashift and non-default recordsize is needed to reproduce this issue.

Test script:

#!/bin/sh

for A in `seq 1 4`; do
    dd if=/dev/zero bs=512M count=1 of=wrongdufile$A iflag=fullblock
done

zpool create -f -o ashift=12 wrongdu raidz `pwd`/wrongdufile*
zfs set recordsize=1M wrongdu
dd if=/dev/urandom bs=16M count=16 of=/wrongdu/testfile iflag=fullblock

ls -lsh /wrongdu/testfile
sync
ls -lsh /wrongdu/testfile
sync
sleep 5
ls -lsh /wrongdu/testfile
du -h /wrongdu/testfile

zpool destroy wrongdu
rm wrongdu*

Result (dd output skipped):

234M -rw-r--r-- 1 root root 256M Feb 21 22:18 /wrongdu/testfile
249M -rw-r--r-- 1 root root 256M Feb 21 22:18 /wrongdu/testfile
249M -rw-r--r-- 1 root root 256M Feb 21 22:18 /wrongdu/testfile
249M    /wrongdu/testfile
edo1 commented 4 years ago

The same result on another host with 0.6.5.9-5 zfs module version.

soyfrien commented 4 years ago

If file compression is enabled, try du with the --apparent-size flag

$ zfs get compression pool/data
NAME           PROPERTY     VALUE     SOURCE
pool/data  compression  lz4       inherited from pool
$ du -hs /data/_smbcheck/windwows-client.txt 
268K    /data/_smbcheck/windwows-client.txt
$ du -hs --apparent-size /data/_smbcheck/windwows-client.txt 
867K    /data/_smbcheck/windwows-client.txt
edo1 commented 4 years ago

@ppdac du --apparent-size shows correct size (like ls -l in my example does). I found that reported allocated size on real pool is fewer than real size even with incompressible files. And I got the same behavior with compression=off (test script doesn't enable compression).

rincebrain commented 4 years ago

You could ask zdb -ddddd [dataset] [object id] to see exactly how ZFS is breaking down the space allocated for a given file.

Object id in this case you can get from ls -i.

edo1 commented 4 years ago

zdb -ddddd [dataset] [object id]

It doesn't work for me:

root@debian:/wrongdu# ls -i
2 testfile
root@debian:/wrongdu# zdb -ddddd wrongdu 2
Dataset mos [META], ID 0, cr_txg 4, 541K, 56 objects, rootbp DVA[0]=<0:300a0000:2000> DVA[1]=<0:380a0000:2000> DVA[2]=<0:500f4000:2000> [L0 DMU objset] fletcher4 uncompressed unencrypted LE contiguous unique triple size=1000L/1000P birth=17390L/17390P fill=56 cksum=213e3e511:7fed0f004d9:f6677e96c9d37:13cbe82f716bfb1b

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
zdb: dmu_object_info() failed, errno 2

I found this object in full zdb -ddddd wrongdu output:

    Object  lvl   iblk   dblk  dsize  dnsize  lsize   %full  type
         2    2   128K     1M   249M     512   256M  100.00  ZFS plain file
                                               176   bonus  System attributes
        dnode flags: USED_BYTES USERUSED_ACCOUNTED USEROBJUSED_ACCOUNTED 
        dnode maxblkid: 255
        path    /testfile
        uid     0
        gid     0
        atime   Sat Feb 22 17:49:57 2020
        mtime   Sat Feb 22 17:49:59 2020
        ctime   Sat Feb 22 17:49:59 2020
        crtime  Sat Feb 22 17:49:57 2020
        gen     9
        mode    100644
        size    268435456
        parent  34
        links   1
        pflags  840800000004
Indirect blocks:
               0 L1  0:3006a000:4000 20000L/3000P F=256 B=14/14 cksum=4f5601518b8:1f58b00260f2ca:7db8a197edcba460:ddd1b49a6675baf7
               0  L0 0:20578000:156000 100000L/100000P F=1 B=9/9 cksum=1ffbb2eff45b0:ffa2de7ad9f0d7b5:3b7d10842131c178:1f10750d3a9c7fd3
          100000  L0 0:20020000:156000 100000L/100000P F=1 B=9/9 cksum=200efa72a92f0:217b78af16ad695:65adfc81d63efea2:570968f851a89175
          200000  L0 0:20824000:156000 100000L/100000P F=1 B=9/9 cksum=2010a1bae7368:1bfba978bb6078f:e0c984eefabc1cdf:168d098bc712965e
          300000  L0 0:20176000:156000 100000L/100000P F=1 B=9/9 cksum=1ffd647c28b53:7aeec21d711db2:38b32d1f9fc16e6e:458fc8c73e3b2531
          400000  L0 0:202cc000:156000 100000L/100000P F=1 B=9/9 cksum=20029fb1c0d5f:ff83cde29c54c1d9:6f8b6aec41df561f:c10e58ee5fe73d9e
          500000  L0 0:2097a000:156000 100000L/100000P F=1 B=9/9 cksum=1ffc3b48e2ac8:ff0b119d42dcaaea:98a75ca0f54215de:df3d39cd5874520d
          600000  L0 0:20422000:156000 100000L/100000P F=1 B=9/9 cksum=1ff8f83453854:ff521b3ea7a0a2f8:75872d5df0bb125e:2e20b71ad56bd845
          700000  L0 0:206ce000:156000 100000L/100000P F=1 B=9/9 cksum=20054ac602519:44c887bcb63ef4:1a39c9c3e1f5ef96:cfbd831039ba74e2
          800000  L0 0:20ad0000:156000 100000L/100000P F=1 B=9/9 cksum=1ffef8f43b013:ae87617004c935:89ed0fdee94c98bd:ab70888cf1241e4e
          900000  L0 0:20c26000:156000 100000L/100000P F=1 B=9/9 cksum=200496b2d80f2:172a0171e8f1cf0:89bcbba966d18b6:675f37d4f645672f
          a00000  L0 0:21028000:156000 100000L/100000P F=1 B=9/9 cksum=2011b4c416724:27384a8367f451d:4e855b17da3ede3b:d7eebab0248eda4f
          b00000  L0 0:212d4000:156000 100000L/100000P F=1 B=9/9 cksum=1ffdc75443d55:fef8780e6f92a934:4453937c3b242937:54d7424cee2fdff6
          c00000  L0 0:2142a000:156000 100000L/100000P F=1 B=9/9 cksum=1ffe0485190a4:ff01de31cb072c6b:6b249930c0e7d06e:10503b2a68dcc4ff
          d00000  L0 0:20ed2000:156000 100000L/100000P F=1 B=9/9 cksum=1fec8f8df842f:fdfa68a323911abf:8438777c59cf60fa:10a1e02ae0e91d26
          e00000  L0 0:2117e000:156000 100000L/100000P F=1 B=9/9 cksum=1fe975b32a43e:fc5326ac5bd78d44:3b008ff30e2f9f0b:ee209476fe30af17
          f00000  L0 0:216d6000:156000 100000L/100000P F=1 B=9/9 cksum=200497570e70b:be3ec28512e690:54ff48ad5e93cc6b:187367cfa00dae2b
         1000000  L0 0:20d7c000:156000 100000L/100000P F=1 B=9/9 cksum=2008c0a6667ed:a10c4c4c9373a1:64685340dea303bc:758f7cbf406c683a
         1100000  L0 0:21580000:156000 100000L/100000P F=1 B=9/9 cksum=1ffba15b6f306:ffbb632502971f26:237d393fef8a3160:b2f7e9828a48be5a
         1200000  L0 0:2182c000:156000 100000L/100000P F=1 B=9/9 cksum=1ff657ac6af88:fe86a3af692cb5b7:e7a4c445e18c6dbd:39ea2a5143db15a1
         1300000  L0 0:21982000:156000 100000L/100000P F=1 B=9/9 cksum=200714d044c4c:14494f8398d8242:56754dab0eae925a:5888efb4b26d4274
         1400000  L0 0:21c2e000:156000 100000L/100000P F=1 B=9/9 cksum=2000141fe5cf4:e6eb2c144db239:f5893936d30b2b24:d38c62e8acf83d34
         1500000  L0 0:21ad8000:156000 100000L/100000P F=1 B=9/9 cksum=20049f342d146:6442475b5b042f:dcbcf03877073c5:e4f9348143c6081b
         1600000  L0 0:22030000:156000 100000L/100000P F=1 B=9/9 cksum=1ff9bca25c241:ff4860842ffc0338:f31cffe7ccd78a99:a25cd463f1266f5b
         1700000  L0 0:21d84000:156000 100000L/100000P F=1 B=9/9 cksum=1ffef175d1df6:ff6284309ba2577a:26041a6af7e7cb12:4596597ccb8c991a
         1800000  L0 0:22186000:156000 100000L/100000P F=1 B=9/9 cksum=1fffaa805e3f1:ff444fc02e02ac59:5b668bfff36ce0db:c7e3f5f84f2b9596
         1900000  L0 0:21eda000:156000 100000L/100000P F=1 B=9/9 cksum=20049f86ce071:5292c956734238:4a49b8671649157:809932d473821f20
         1a00000  L0 0:22432000:156000 100000L/100000P F=1 B=9/9 cksum=1ff6b8c145b64:ffaf9c690b99a49c:a7d7830a8dfaa77:c266bd294b78c80f
         1b00000  L0 0:222dc000:156000 100000L/100000P F=1 B=9/9 cksum=1ffa4ab8850cf:fe9a4acdaa7a3653:d912d90595c05cf:82a34f65bb533039
         1c00000  L0 0:22588000:156000 100000L/100000P F=1 B=9/9 cksum=200a811d3325b:1cf9c344d9387b4:c41b7298d5fc0987:689636364d60324e
         1d00000  L0 0:2298a000:156000 100000L/100000P F=1 B=9/9 cksum=1ff509128aea1:fe5823a2470a65c7:64fb6892c021846e:24fa88396917ea20
         1e00000  L0 0:22ae0000:156000 100000L/100000P F=1 B=9/9 cksum=201447fb3ece1:324b0950e66d3a8:346417ed910da532:84bdb3a2651d9f75
         1f00000  L0 0:22834000:156000 100000L/100000P F=1 B=9/9 cksum=1ffda4167c8a3:ffa6c679ab5bc57f:23324950a5018de7:9c73d32d9fdfb7f5
         2000000  L0 0:226de000:156000 100000L/100000P F=1 B=9/9 cksum=20043a28412dd:e141b12723d52e:546af286d555ba24:fae11a05e652242b
         2100000  L0 0:22ee2000:156000 100000L/100000P F=1 B=9/9 cksum=200b7230c033a:1584618f9333369:6f3b91a6d4446a20:dd108ec066e1bb72
         2200000  L0 0:22c36000:156000 100000L/100000P F=1 B=9/9 cksum=200925c4d56cd:877dce649cd654:5583f8230cafefb9:e5d2e73c1b11efb3
         2300000  L0 0:22d8c000:156000 100000L/100000P F=1 B=9/9 cksum=2008218ca7c10:c7ce5033fa45cb:d448b9a634188143:377807394fee3318
         2400000  L0 0:2318e000:156000 100000L/100000P F=1 B=9/9 cksum=1fe89e241c913:fd79cdee7db61082:f5a3d98d5763110e:aaffdb045bfecc9b
         2500000  L0 0:23038000:156000 100000L/100000P F=1 B=9/9 cksum=1fecc7fc63f1f:fe6908367d0425cb:4360d8fbe049a0bf:299e4a48d91c9475
         2600000  L0 0:232e4000:156000 100000L/100000P F=1 B=9/9 cksum=1ff2c9f2df5a3:feb8a1448b83d91a:c8ec2259bb347e0b:e7c57451604729f0
         2700000  L0 0:2343a000:156000 100000L/100000P F=1 B=9/9 cksum=20006c1b5519c:ff54c1686b66ff6c:2c815940d852f265:8629def5b8b6b4e2
         2800000  L0 0:236e6000:156000 100000L/100000P F=1 B=9/9 cksum=200e5a4cb98c8:19d1b07e29841d2:b78d654c2d1f273d:6984d85805327975
         2900000  L0 0:23590000:156000 100000L/100000P F=1 B=9/9 cksum=1ffea94be6cf4:fff40efdc30ac970:e746543384c2e9bd:da34bb3ec07c3a3a
         2a00000  L0 0:2383c000:156000 100000L/100000P F=1 B=9/9 cksum=1ff181880a3fa:fd65f20ffbb1051d:9db29c57f5faf301:12989ea49a42bff1
         2b00000  L0 0:23992000:156000 100000L/100000P F=1 B=9/9 cksum=1ffae1acea47a:ffd198680e9b61a1:2ccc3d356ffc8f95:ab2575dc60b16234
         2c00000  L0 0:23ae8000:156000 100000L/100000P F=1 B=9/9 cksum=1ff2627bbc5f0:fd6a9d94b9231aa0:1ff12225e57628e1:b6560a84440e8d3c
         2d00000  L0 0:23c3e000:156000 100000L/100000P F=1 B=9/9 cksum=1ffeb1588e179:ca251a8fd5eab5:de789071c77bd055:4da60ade77096a4a
         2e00000  L0 0:23d94000:156000 100000L/100000P F=1 B=9/9 cksum=201ed463f1399:3dea33c4ec2009b:497bf6cfb440868d:5f2bb41095aa0c
         2f00000  L0 0:23eea000:156000 100000L/100000P F=1 B=9/9 cksum=2006ca5e3fb46:c18050de7ee6f4:12fa781d07526ab:db97858a660163af
         3000000  L0 0:48000000:156000 100000L/100000P F=1 B=9/9 cksum=200256ac7476b:b4116a283b7e8:e44b8a174e8aa017:ea97ee9115f69554
         3100000  L0 0:48156000:156000 100000L/100000P F=1 B=9/9 cksum=200115a9bac5d:118f632185dd6b9:edcf590dd71de1b6:7cd9880bb13e3268
         3200000  L0 0:482ac000:156000 100000L/100000P F=1 B=9/9 cksum=1ffedb3172a28:81f86600d260b:4a37ad2e55f19ccd:e0568e8c05954b6a
         3300000  L0 0:48402000:156000 100000L/100000P F=1 B=9/9 cksum=1fff0fdb19af9:f543201243c7e3:801112e8e71a6352:b746f7cbb595a8f2
         3400000  L0 0:48964000:156000 100000L/100000P F=1 B=10/10 cksum=2003a7cb1cdf9:b7c20b903f269d:b0dda1120504b1d1:1f597076715b0466
         3500000  L0 0:48562000:156000 100000L/100000P F=1 B=10/10 cksum=1ff508e0e8440:feb34fd78cc3619a:e120371033995a4:8ae9a2ca2b96a8c3
         3600000  L0 0:486b8000:156000 100000L/100000P F=1 B=10/10 cksum=200988a45c9d5:21dd55ba36f3d45:3ca17049d90d14b7:93dac33f9da29d0c
         3700000  L0 0:49168000:156000 100000L/100000P F=1 B=10/10 cksum=1ffe750e0d028:982c498c04e156:6638f466593e06cb:f5a6f46c17e9719f
         3800000  L0 0:492be000:156000 100000L/100000P F=1 B=10/10 cksum=1fff9d549107e:ff2b03949edef426:eb3a04b4ce74d1e8:8e3592658c16d066
         3900000  L0 0:48d66000:156000 100000L/100000P F=1 B=10/10 cksum=20018b79aedc7:d96c47cd4115be:f188d1eda87ae5bb:f7aadcdd4d6495e0
         3a00000  L0 0:48ebc000:156000 100000L/100000P F=1 B=10/10 cksum=20093c805570e:e2471bdc62cda5:606aa01c0cf21186:d0874d18d64a268c
         3b00000  L0 0:4880e000:156000 100000L/100000P F=1 B=10/10 cksum=2000652c1c756:ffce2a33fce1501a:1498d3c154ccb8f9:717b107a01abe68
         3c00000  L0 0:49414000:156000 100000L/100000P F=1 B=10/10 cksum=1ff8e945719c5:fff6fc6f70163ef5:a838509c31c0f1fb:6433592a5c1c5ddb
         3d00000  L0 0:48aba000:156000 100000L/100000P F=1 B=10/10 cksum=1ffd2da0e6f30:ff32a485bb8608ba:2ef2ccc3ae3e0a15:7aa925d4f32b9411
         3e00000  L0 0:48c10000:156000 100000L/100000P F=1 B=10/10 cksum=1fee5c8436668:ff235252fa5d5cfd:396d66684a0bc9fc:84191e309715d1d5
         3f00000  L0 0:49012000:156000 100000L/100000P F=1 B=10/10 cksum=2003d51f727ae:fa1a6b9d26340c:35d8fdfd12a425c2:ca35f2430334f403
         4000000  L0 0:496c0000:156000 100000L/100000P F=1 B=10/10 cksum=1ffa0edf42a2e:ffa230db9fdc10cf:39b3585f3000f91f:ff404f05dacbb731
         4100000  L0 0:49816000:156000 100000L/100000P F=1 B=10/10 cksum=201ce178783d1:2fc197a37f7fd69:6a4cfba398349de9:2e39acdc44da23ab
         4200000  L0 0:4956a000:156000 100000L/100000P F=1 B=10/10 cksum=200c465142dbb:13511cf48b2c741:43881cb48d64118a:7b3247d361a8b0f3
         4300000  L0 0:4996c000:156000 100000L/100000P F=1 B=10/10 cksum=1fff4c62df6f4:ffed23d5d8f1cf78:dbf6b7f9b0f68c0:2cfcd8f15f85df82
         4400000  L0 0:49ac2000:156000 100000L/100000P F=1 B=10/10 cksum=2003b358eca30:1af2d2e4dc86666:a367294f1c190113:f40449e47bb5c575
         4500000  L0 0:49c18000:156000 100000L/100000P F=1 B=10/10 cksum=1ff8e41de3293:fef1ca67993d2ed3:48d3cb41409e3e70:aea754a73b675325
         4600000  L0 0:49d6e000:156000 100000L/100000P F=1 B=10/10 cksum=2014f0d605769:38ed143a4c6630e:ac884c43c1f6029f:cff29ecaffbe1038
         4700000  L0 0:49ec4000:156000 100000L/100000P F=1 B=10/10 cksum=200032dd140df:ffead8246c73de87:b658de22f122fae9:9869fa5228179923
         4800000  L0 0:4a170000:156000 100000L/100000P F=1 B=10/10 cksum=1fec9374b14c0:fce2a10f62b22ebf:d959cf149e01c0e1:237478d25a1359af
         4900000  L0 0:4a01a000:156000 100000L/100000P F=1 B=10/10 cksum=1ff241054ce0b:fd6c7d7b67827d5c:34d05f15c4f9f4ed:85d1cc6361df96f2
         4a00000  L0 0:4a2c6000:156000 100000L/100000P F=1 B=10/10 cksum=20033cb683069:3cd89b0d10968d:3e6055f1dd556cc8:a975e2421033e336
         4b00000  L0 0:4a41c000:156000 100000L/100000P F=1 B=10/10 cksum=2008c3636f302:1c5180de95eee82:ef7e776abffc2be6:e83fa0a21b6d1b47
         4c00000  L0 0:4a572000:156000 100000L/100000P F=1 B=10/10 cksum=1ffe611a2fb06:ffe9b4cdaa3713f4:f428365a466689a5:5414038cd839c504
         4d00000  L0 0:4a6c8000:156000 100000L/100000P F=1 B=10/10 cksum=1ffa90ef73a44:fefefdae7d1efea6:c8fbd04c65eb249e:33d0d292cfb99bb2
         4e00000  L0 0:4a81e000:156000 100000L/100000P F=1 B=10/10 cksum=200126d59575a:ff1aae6fc7d22600:662e83f93a7f84ff:84c746045848e30a
         4f00000  L0 0:4a974000:156000 100000L/100000P F=1 B=10/10 cksum=1ffa5866da11a:feee1c7070c910fc:60f6e6a63ab24488:a7a9a716b6e4e6da
         5000000  L0 0:4aaca000:156000 100000L/100000P F=1 B=10/10 cksum=200b97fb877d9:ec4459de1f4676:af4ac6b86d98c70c:f8ac206e5c1c96f2
         5100000  L0 0:4b178000:156000 100000L/100000P F=1 B=10/10 cksum=1ffc33d3cc46e:ff27937692742d48:df15ca658736d952:5e35020d76099012
         5200000  L0 0:4ac20000:156000 100000L/100000P F=1 B=10/10 cksum=200b3ec1c1dfc:189093320ae0e4a:c18746b66fb0dd13:5c5883212119e8e4
         5300000  L0 0:4ad76000:156000 100000L/100000P F=1 B=10/10 cksum=1ff80db9bd7a1:ff66547f3a7b38d4:350efa25e443eb78:32f4b495b9627df6
         5400000  L0 0:4b022000:156000 100000L/100000P F=1 B=10/10 cksum=1ffbbaa125d7d:ff70fc33e4f69b30:eba738998a93c974:c117221b8040c728
         5500000  L0 0:4b57a000:156000 100000L/100000P F=1 B=10/10 cksum=200201fab69af:ff71803e1c84887e:6a718c8bfbb3e913:2840e84e7d852867
         5600000  L0 0:4aecc000:156000 100000L/100000P F=1 B=10/10 cksum=1ffbe7e462520:ff1395d246e7a638:55d9bff178f99946:ab529c410b0cf82f
         5700000  L0 0:4b2ce000:156000 100000L/100000P F=1 B=10/10 cksum=2006eafe35394:1b04c513f5c8515:b97078e1240632f7:94d668c317dbd62
         5800000  L0 0:4b424000:156000 100000L/100000P F=1 B=10/10 cksum=1feb3c259909d:fd98950837005af2:7c7bcad01f8f55a9:bb192f547d78f783
         5900000  L0 0:4b6d0000:156000 100000L/100000P F=1 B=10/10 cksum=1ff829bc4c2a5:ff71dc487090eb77:e538bdde5576b47e:a244afff90627bd
         5a00000  L0 0:4b826000:156000 100000L/100000P F=1 B=10/10 cksum=20066d3d751e0:6cf81c456c16f4:4d2c42319b44f87b:c9ef65a90f7410e2
         5b00000  L0 0:4b97c000:156000 100000L/100000P F=1 B=10/10 cksum=2006ff31011ff:c3a9e217de0e37:ccc5e61ef5aefc33:284a2d4016198654
         5c00000  L0 0:4bad2000:156000 100000L/100000P F=1 B=10/10 cksum=1fea0b291e54f:fcd02864a1af6442:740185739ecd861a:609d3499d5fb6ea7
         5d00000  L0 0:4bc28000:156000 100000L/100000P F=1 B=10/10 cksum=1ffe5d2749002:febbd58fb0617aa6:aa09dc8ada8adc03:3dd9193e40ef9288
         5e00000  L0 0:4bed4000:156000 100000L/100000P F=1 B=10/10 cksum=1ff8f00e6e011:fff03d74dd9be459:549ad23b79bfe88f:93f5a821feeb117f
         5f00000  L0 0:4bd7e000:156000 100000L/100000P F=1 B=10/10 cksum=2000c50e3e300:ff602a4a8a4c087d:248022aee2e4c110:d5725455ee6deceb
         6000000  L0 0:58000000:156000 100000L/100000P F=1 B=10/10 cksum=1ff8a7fcb0aa0:4af9472471d6b2:81037f1333d5b38b:a8ff9644f453ad5a
         6100000  L0 0:58156000:156000 100000L/100000P F=1 B=10/10 cksum=2001beeabcd3a:ff510d0618c5d8b7:132c7d451ec9bdbb:20c1f192fb490e54
         6200000  L0 0:582ac000:156000 100000L/100000P F=1 B=10/10 cksum=2004156bba89a:ffa7ba9ff61ab55c:aa8298fca351da88:a5ddcf11ba4f23ed
         6300000  L0 0:58402000:156000 100000L/100000P F=1 B=10/10 cksum=20013a09c7955:b9c1969150d994:a82f0888c0a14c81:6e85e909d7d55bbf
         6400000  L0 0:58558000:156000 100000L/100000P F=1 B=10/10 cksum=1fffee24531ee:ffb14f70d0c7ea4e:93f062ced51f55fc:4aaa1a7581f41192
         6500000  L0 0:586ae000:156000 100000L/100000P F=1 B=10/10 cksum=2002f7f580605:896fd58ab54b5a:bbabe81ba331302a:47099e0cd56933e0
         6600000  L0 0:58d68000:156000 100000L/100000P F=1 B=11/11 cksum=20029f9de8a9c:6de7397ca5a4d4:5a1224a3ca5225c:79675263741ebaa
         6700000  L0 0:58810000:156000 100000L/100000P F=1 B=11/11 cksum=1ffef001291b0:fe4cbf0969ba1705:3a8c889bff81756b:68c981f37439b3b4
         6800000  L0 0:58966000:156000 100000L/100000P F=1 B=11/11 cksum=2017297569a89:3f695143937c0b3:979b9f6c7579cba:280e8953d0c64faf
         6900000  L0 0:58abc000:156000 100000L/100000P F=1 B=11/11 cksum=1ffee1a03615f:ffd14bddc143aff0:f318e97f34f428e2:f3225a3b3877817e
         6a00000  L0 0:58c12000:156000 100000L/100000P F=1 B=11/11 cksum=1ffdae31abf33:d3a0fea35efc2a:175b23113969585f:409c7f4211f16483
         6b00000  L0 0:59014000:156000 100000L/100000P F=1 B=11/11 cksum=200107ebcd5fe:567ed932952f12:b984d70f132fc71:47e619eb4b727974
         6c00000  L0 0:5916a000:156000 100000L/100000P F=1 B=11/11 cksum=200be4cabb6e2:281fa1589c323ad:fd0bd6ad743a19fb:a4dd09f5544e0817
         6d00000  L0 0:59416000:156000 100000L/100000P F=1 B=11/11 cksum=200c572679db0:b862ba32744cc2:8322d46c93348c88:6f68600bf17235c
         6e00000  L0 0:58ebe000:156000 100000L/100000P F=1 B=11/11 cksum=20000fa3d6560:ff3f8c156b139e88:5aaf194ccbbf7c62:acbbebed3ab6231e
         6f00000  L0 0:592c0000:156000 100000L/100000P F=1 B=11/11 cksum=1ff84734513f0:fdb46a5160ccce85:a66db748ce0e9989:b6fff0a5410a32e8
         7000000  L0 0:5956c000:156000 100000L/100000P F=1 B=11/11 cksum=1ff61a6755476:fe77b92081defa19:6367299f890e99d6:d51b62841c93984f
         7100000  L0 0:5996e000:156000 100000L/100000P F=1 B=11/11 cksum=2007b7166b7da:f6599bdcac103d:d39d8a794c74351b:eeb5d5ed05a9e0fc
         7200000  L0 0:59ac4000:156000 100000L/100000P F=1 B=11/11 cksum=1ffa8e8a26a61:ff7be096579ee5a1:33e7ccc0f098afdb:349b9a056bd5965a
         7300000  L0 0:596c2000:156000 100000L/100000P F=1 B=11/11 cksum=1ff56ec0021f8:feb5761579d653ff:7a915a48a6cc4797:6d91abf778cdea70
         7400000  L0 0:59818000:156000 100000L/100000P F=1 B=11/11 cksum=1ffddc1063a60:ffc0cd2feb4475a0:962bc5b132f8391e:6b333c318de43b1d
         7500000  L0 0:59c1a000:156000 100000L/100000P F=1 B=11/11 cksum=1ffcf26fbd3d3:fee4e0f820405f52:287e4d7c7b247fc4:c0531aa7b43580ea
         7600000  L0 0:59ec6000:156000 100000L/100000P F=1 B=11/11 cksum=1ffb6bc719c5d:fed3d0b1cc3a9ff7:72d1cfa9dde6c4a9:955c60ce4522f1e0
         7700000  L0 0:59d70000:156000 100000L/100000P F=1 B=11/11 cksum=1fff06ba303b5:5508863b0074c9:a0f2f396c576279a:4bb92d7abebfe640
         7800000  L0 0:5a01c000:156000 100000L/100000P F=1 B=11/11 cksum=1ff8dd1862f74:ff70ecc61a50f2e7:ffa7d6c5308093d5:2f4518b6253b92
         7900000  L0 0:5a172000:156000 100000L/100000P F=1 B=11/11 cksum=1ffbf5004ce12:ffc3bed66467fd94:962b29aaa20e6555:f32585680fc38fcb
         7a00000  L0 0:5a2c8000:156000 100000L/100000P F=1 B=11/11 cksum=2006ac1f4784b:f283acc3b845f9:c2e7bfb5d6bbb170:1a6737f9a6c182f
         7b00000  L0 0:5a41e000:156000 100000L/100000P F=1 B=11/11 cksum=20095e4241613:baa53414f83bf8:6ccd3553bcbab44a:564f64d5cc062298
         7c00000  L0 0:5a820000:156000 100000L/100000P F=1 B=11/11 cksum=1ffc7dcc59a37:ff27be84597daa53:cb91c48c42ebcef6:f667ebd3deabf05
         7d00000  L0 0:5a6ca000:156000 100000L/100000P F=1 B=11/11 cksum=1ff60d91cb208:fe5332f4e143f7b7:3bffa1e43c33c8ec:784241be6ad2a03a
         7e00000  L0 0:5a976000:156000 100000L/100000P F=1 B=11/11 cksum=200ccdf68f8c1:1a4e8f2ed07eb2d:60b5bdf617b1a1e8:1d3067a716ed1898
         7f00000  L0 0:5a574000:156000 100000L/100000P F=1 B=11/11 cksum=1ffaa5853bae6:10e6e0a43ed2162:5b49e449631be4d6:f99fd5e51e4232c9
         8000000  L0 0:5ac22000:156000 100000L/100000P F=1 B=11/11 cksum=1ffbd69c7716b:fe89482e490efa38:8340993e8bcf5ca0:3ade7067eab377a5
         8100000  L0 0:5ad78000:156000 100000L/100000P F=1 B=11/11 cksum=1ffadfb9c0be9:fe163f3b6109c8cb:48f62bc020e50d83:9924cbd57f29ed8d
         8200000  L0 0:5aece000:156000 100000L/100000P F=1 B=11/11 cksum=2001c148cde0c:ff9a80458c712dd4:839dc2108d812202:df5dd92eeaf46758
         8300000  L0 0:5aacc000:156000 100000L/100000P F=1 B=11/11 cksum=1ffe81a35c62d:ffc1cd7f710a937c:e44f800c998da3d3:bf487d36bb562d44
         8400000  L0 0:5b024000:156000 100000L/100000P F=1 B=11/11 cksum=200f88cade6be:26191936023755a:c23fd73750ea854c:609f1725c9ff670e
         8500000  L0 0:5b2d0000:156000 100000L/100000P F=1 B=11/11 cksum=1ffd19677d8ca:9ba244cf7eb93c:fc3fb2ac7c84aa61:585c135b9227b40e
         8600000  L0 0:5b17a000:156000 100000L/100000P F=1 B=11/11 cksum=1ff900da3e2a7:ffdc5bf6396a66f6:3ce8f5ea5c9b3bbb:edf3c79d3e1b89db
         8700000  L0 0:5b6d2000:156000 100000L/100000P F=1 B=11/11 cksum=1ffb78b76a55d:ffae676e692808d1:e32b3064889cbf85:2e659419e6f3ac43
         8800000  L0 0:5b57c000:156000 100000L/100000P F=1 B=11/11 cksum=2001b9f60de4f:7adcc3b5c099b4:105465555b896cbd:acc8b9e8026a30b4
         8900000  L0 0:5b426000:156000 100000L/100000P F=1 B=11/11 cksum=2002cd9a62bee:11e39bbcc08a4d0:eed1a6fc9b7209f9:591f3fa7e3d45598
         8a00000  L0 0:5b97e000:156000 100000L/100000P F=1 B=11/11 cksum=2012070b94ff3:25cd1627f99b24d:4eb68b47d994f828:25560dbe43e9857b
         8b00000  L0 0:5b828000:156000 100000L/100000P F=1 B=11/11 cksum=1ff84ee449ba5:fe53d936a685457d:4a2b92be8397d750:eaff37be1413862
         8c00000  L0 0:5bad4000:156000 100000L/100000P F=1 B=11/11 cksum=1ff720d976073:ff869283b1fa205d:fa6d0647d87b8877:be33fbefac097733
         8d00000  L0 0:5bc2a000:156000 100000L/100000P F=1 B=11/11 cksum=2008f06d98245:28b32727412592d:c6c8c77574d4c799:87706a28ecb27a08
         8e00000  L0 0:5bd80000:156000 100000L/100000P F=1 B=11/11 cksum=1ff751276bbc6:fda67adb7819791c:56229cd8b01b5ee2:ac89e3128506ae4d
         8f00000  L0 0:5bed6000:156000 100000L/100000P F=1 B=11/11 cksum=20008ea55f5a3:137d1e39db1ba8:bc3e18c9af4023f:812609f0c8b755bc
         9000000  L0 0:68000000:156000 100000L/100000P F=1 B=11/11 cksum=1ffae85fa4440:ff5c053d077aeca9:b8cd28fbcebee21e:9488d776b7394890
         9100000  L0 0:68156000:156000 100000L/100000P F=1 B=11/11 cksum=1ff4b5cf05996:fe2cce48808a2e57:a8fd487f5bd2e3d9:8a26662cb78fbe99
         9200000  L0 0:682ac000:156000 100000L/100000P F=1 B=11/11 cksum=200ea4c88c29d:35893ce219800ad:a420fe09f3203f2b:f484228b8f2655ed
         9300000  L0 0:68402000:156000 100000L/100000P F=1 B=11/11 cksum=200193220225a:3b2cd24936638e:b4d17e9fa1d280cc:d74b76467794a5cc
         9400000  L0 0:68558000:156000 100000L/100000P F=1 B=11/11 cksum=1ff08bb0934ad:fe78163b4c130cfe:5f5fb9ea3eb38248:739cbef07d75d592
         9500000  L0 0:686ae000:156000 100000L/100000P F=1 B=11/11 cksum=2001d83b0f94f:ff784008ab7fe1df:aebda485968bfe1a:ee1ef9d8c8663a16
         9600000  L0 0:68968000:156000 100000L/100000P F=1 B=12/12 cksum=2004a312e6944:18b5e3d595249bc:267727efc68b354a:d12474d1fce6e98b
         9700000  L0 0:68c14000:156000 100000L/100000P F=1 B=12/12 cksum=1ffc0fff7edab:1b70a4bbe90157:7658fa403f73fcb:576d5cd402cdd877
         9800000  L0 0:68abe000:156000 100000L/100000P F=1 B=12/12 cksum=1ff1282868844:fe490999d774b885:1fec192ef26f2b9a:28ac1e7f42d0f394
         9900000  L0 0:68ec0000:156000 100000L/100000P F=1 B=12/12 cksum=1ffc11c4059ec:ffb2757c0f1334c8:3db174d7205a5f83:c8432c3e53f69b7c
         9a00000  L0 0:69016000:156000 100000L/100000P F=1 B=12/12 cksum=1fee3d991c763:fe0b45a5ac8f6537:d45dfa8f5b0dffc4:fe6f8d57be365e69
         9b00000  L0 0:68812000:156000 100000L/100000P F=1 B=12/12 cksum=200374b84763c:fefd5884d11b11f2:e004568592ebca44:e8f289014595621e
         9c00000  L0 0:6956e000:156000 100000L/100000P F=1 B=12/12 cksum=1ff07a1da94d3:fedd97bd81aa3a3d:edc31bd71c6f10a:55c3142582d78d1c
         9d00000  L0 0:696c4000:156000 100000L/100000P F=1 B=12/12 cksum=1fe9a01c21fe0:fceb18dcc78ac099:cb75bb927e83d540:d44ca34bb1ec5f35
         9e00000  L0 0:69418000:156000 100000L/100000P F=1 B=12/12 cksum=1ffa9e1917f85:fef37d38628fb27e:d4dd10f16b7c3b57:8134780b7f0d8e81
         9f00000  L0 0:69970000:156000 100000L/100000P F=1 B=12/12 cksum=20051001e2649:148ee4457a537f5:49b472f6c2e4df98:fab55d80636847a6
         a000000  L0 0:6981a000:156000 100000L/100000P F=1 B=12/12 cksum=1ffc73919cd29:fea93e7eb1ae3177:739f1b13d6c94505:bfb1a0dc88a01352
         a100000  L0 0:68d6a000:156000 100000L/100000P F=1 B=12/12 cksum=1ff62ff2a6991:fe9f01862416cae3:a80e77c71cff8bfa:5ef4fc2ce931f756
         a200000  L0 0:69ac6000:156000 100000L/100000P F=1 B=12/12 cksum=2014897e8d89a:33e30ebadc51cde:5cbf4ce6242749e0:9e44d1ff75ce87cc
         a300000  L0 0:6916c000:156000 100000L/100000P F=1 B=12/12 cksum=201290bc10fa1:24bd34fe21a6fc9:e00ea570d591aaa1:daeaf17805542370
         a400000  L0 0:69c1c000:156000 100000L/100000P F=1 B=12/12 cksum=200197bc57aa1:ae430b73e17292:afe4e84e3f7ef5c1:77c64d637e9ea7b9
         a500000  L0 0:692c2000:156000 100000L/100000P F=1 B=12/12 cksum=1ffd0bf3c3a1d:3fdf63a199149:4828581cf4db3587:d05a6548f5da2635
         a600000  L0 0:69ec8000:156000 100000L/100000P F=1 B=12/12 cksum=20001f1ad8bf5:ffd958a22a7802a0:6492b301282fb436:41e56ee7bbb801b0
         a700000  L0 0:69d72000:156000 100000L/100000P F=1 B=12/12 cksum=200480132484e:fe9330e3053e7e44:7ac6768da2d91d63:54767cd4509ede0
         a800000  L0 0:6a01e000:156000 100000L/100000P F=1 B=12/12 cksum=1ff55b276fb43:ff32112b9156c6ce:ab6b9128da672200:7a85ac820eefc596
         a900000  L0 0:6a174000:156000 100000L/100000P F=1 B=12/12 cksum=1ffbf768f037f:ff2e46b33b736a3a:b81796e574594598:b5eaba8c2637d728
         aa00000  L0 0:6a2ca000:156000 100000L/100000P F=1 B=12/12 cksum=1ff5b675227a5:fdbd481053da5e58:e1b24db1fbf5592e:e73f54fb80593fc0
         ab00000  L0 0:6a420000:156000 100000L/100000P F=1 B=12/12 cksum=200c36e32d739:2419b0d8353400f:a1db3495200e58cb:f3949b942ef7052a
         ac00000  L0 0:6a576000:156000 100000L/100000P F=1 B=12/12 cksum=1ff6608517437:ff5ac9c455f956dc:3e95c63c43b74ee7:28aa4ec3a42a9131
         ad00000  L0 0:6a6cc000:156000 100000L/100000P F=1 B=12/12 cksum=1ff8cdb968a12:fe5a05a682f6108c:fb6debc467d7419e:1895901ece7cefe2
         ae00000  L0 0:6a822000:156000 100000L/100000P F=1 B=12/12 cksum=1ffec99db46bb:ff9e3e6939e432fe:68bf8d29d3602954:e5af62db972cd98c
         af00000  L0 0:6aace000:156000 100000L/100000P F=1 B=12/12 cksum=1ffbf4f84f8ec:ff5e0f8ef7fb7160:1911be43966664ae:66f463ad3edbfb72
         b000000  L0 0:6a978000:156000 100000L/100000P F=1 B=12/12 cksum=1ffe19f1f411f:ffa26d90bb26af1d:791c1264f413f20c:2b4d5e965bbfb438
         b100000  L0 0:6ad7a000:156000 100000L/100000P F=1 B=12/12 cksum=1ffbb4f87cd18:fec6706d04aa334d:33c41d1b16f8c7fe:187c6b945bf2537c
         b200000  L0 0:6b026000:156000 100000L/100000P F=1 B=12/12 cksum=1ffd37b7e2efd:ff3480b9b284e8a2:acf53901b2c028:bac22e5277bb5b70
         b300000  L0 0:6ac24000:156000 100000L/100000P F=1 B=12/12 cksum=2009be5f45bb7:11f84806f3f45e5:12e7543caca97230:20683f4861dd5a2d
         b400000  L0 0:6aed0000:156000 100000L/100000P F=1 B=12/12 cksum=1ff737fbaa3e6:fea40fc7b6539095:7bb30599c912401e:4c5b70a452a8ad1a
         b500000  L0 0:6b17c000:156000 100000L/100000P F=1 B=12/12 cksum=1ff6e4a3445e8:ff518df9ffebf433:f82df64b8f9c30c7:eb75eb77a518c7fb
         b600000  L0 0:6b2d2000:156000 100000L/100000P F=1 B=12/12 cksum=1fee829fd1207:fc744e471cbeba37:c817e3833d99800b:47cd9c15c825d14f
         b700000  L0 0:6b428000:156000 100000L/100000P F=1 B=12/12 cksum=200e80df3ac6e:27c48780923158d:690badb5993c19f:e61d38ec6bda6c05
         b800000  L0 0:6b57e000:156000 100000L/100000P F=1 B=12/12 cksum=1ffc89118c5b0:ffdb2c79d5ea50b2:884fe8bfc01102d6:a64f3074c3398749
         b900000  L0 0:6b6d4000:156000 100000L/100000P F=1 B=12/12 cksum=1ffac7c474943:ffd320daa8f62185:d3501a4003ae2834:13dd127b7529cea9
         ba00000  L0 0:6b82a000:156000 100000L/100000P F=1 B=12/12 cksum=1fe89a380a392:fce7d8618531cd63:2cd9e2182caa0d00:498679f477b4c63
         bb00000  L0 0:6b980000:156000 100000L/100000P F=1 B=12/12 cksum=1ffba24031891:fffd62bb30c968e0:859bc06a834eeadd:9b6e22d33e0e844b
         bc00000  L0 0:6bad6000:156000 100000L/100000P F=1 B=12/12 cksum=201023585d838:2b03e544dfe58a4:4155b6a7d797322a:e95d9207005c77ae
         bd00000  L0 0:6bc2c000:156000 100000L/100000P F=1 B=12/12 cksum=1ffe1f95b38da:ff75ea33740ceb3a:c69811e1785d1e2:3f0d87f3a361109a
         be00000  L0 0:6bd82000:156000 100000L/100000P F=1 B=12/12 cksum=1ffec34c71f5d:ff7f4a98fb0bf55b:2752e9c22d72fd89:a2bce1a9dbb317de
         bf00000  L0 0:40066000:156000 100000L/100000P F=1 B=12/12 cksum=1ffd150ded3a3:ffb17d8ca27ca0be:d7423d82cca928cf:edfba47c5b09b259
         c000000  L0 0:40312000:156000 100000L/100000P F=1 B=12/12 cksum=1ff3d586ce47a:fd0dd571e800328a:cc92902ee65e93ec:d8c4b046ef54cf2
         c100000  L0 0:6bed8000:156000 100000L/100000P F=1 B=12/12 cksum=1fffb02de214a:ffbcfacd837dd34a:9412ac9e6ffe6f28:321a4277bb89534a
         c200000  L0 0:401bc000:156000 100000L/100000P F=1 B=12/12 cksum=1ffec00243604:ffe5302b6f61f2d4:e42f6ef577c44665:cc948cd014530b3
         c300000  L0 0:40468000:156000 100000L/100000P F=1 B=12/12 cksum=20000a34ac6bc:ffb349e7a78a67a9:a67ac58f013df43b:79754d602b4e6db9
         c400000  L0 0:405cc000:156000 100000L/100000P F=1 B=13/13 cksum=1ffd14b7af870:ff568cefccc54a02:112d3c5bfa9642fb:de8d04eacef4ea06
         c500000  L0 0:40878000:156000 100000L/100000P F=1 B=13/13 cksum=2014311d86902:1da420f2e406d36:38fcc17de186b7f3:706a4186c9cfdb6
         c600000  L0 0:40c7a000:156000 100000L/100000P F=1 B=13/13 cksum=1ff949027e9e0:ff6b6716a75cedbd:82cca946444552c8:b219e0b8899e592a
         c700000  L0 0:409ce000:156000 100000L/100000P F=1 B=13/13 cksum=1ff2b268c0536:feea9edcc9270e66:1034cc50e0d0d96f:9cf50609a96477b1
         c800000  L0 0:40dd0000:156000 100000L/100000P F=1 B=13/13 cksum=200382d0767af:dbb13c2365f21c:ce9fc733e5f1596a:27168dc301e64721
         c900000  L0 0:411d2000:156000 100000L/100000P F=1 B=13/13 cksum=200ff080a878f:1afd53adc81a58a:7e89f0244f9e9e02:fa381e73c880e5a6
         ca00000  L0 0:4107c000:156000 100000L/100000P F=1 B=13/13 cksum=1ffeeebcb66f4:470a65627219df:5444e25a0b8a7963:3ef5a74d2428afff
         cb00000  L0 0:40722000:156000 100000L/100000P F=1 B=13/13 cksum=2003f4287f49a:ff63f57702a44f5c:df758e925236570e:4dc644e699d3b9ef
         cc00000  L0 0:40f26000:156000 100000L/100000P F=1 B=13/13 cksum=20111069aab53:1b4e4f863c2c1ac:75d2c012010afc1a:d9b50b94f40e09a5
         cd00000  L0 0:41328000:156000 100000L/100000P F=1 B=13/13 cksum=20167c97a384a:2b6db18d3a2769e:229234365cf45e5e:877b8293d8232102
         ce00000  L0 0:40b24000:156000 100000L/100000P F=1 B=13/13 cksum=1ffc648fb31fc:ff4839a451d5046c:c83c054750297a1a:de50bfcdb7bf4506
         cf00000  L0 0:415d4000:156000 100000L/100000P F=1 B=13/13 cksum=1ff6c00efaad0:fe242b51bb16cd2a:2e985175be72f710:a231cf1bd432bc05
         d000000  L0 0:41880000:156000 100000L/100000P F=1 B=13/13 cksum=200a6d302a5dd:137e38f39afbb49:1d5012ca57d15a03:90d54be044bb48f9
         d100000  L0 0:419d6000:156000 100000L/100000P F=1 B=13/13 cksum=200a491a75a0c:1563e3d92f94caa:2eca8f80c8209d35:b0a74a5818ef67a1
         d200000  L0 0:4147e000:156000 100000L/100000P F=1 B=13/13 cksum=1ff58f6523e41:fe97634928f56334:9bae295ec6cc75b0:1e3ec65ca16cfcf2
         d300000  L0 0:41b2c000:156000 100000L/100000P F=1 B=13/13 cksum=1ff9b671e1dec:fe3594b416d99c92:77f38471eaddf652:50398c11e3e5f88d
         d400000  L0 0:41c82000:156000 100000L/100000P F=1 B=13/13 cksum=1ff425e1afe9b:ff0920b63f36d537:444638ff5b6a19fe:b5d6a410a372ee22
         d500000  L0 0:4172a000:156000 100000L/100000P F=1 B=13/13 cksum=2002ef228e0ea:c562d5b32b07a2:f546c0e428001ac1:4e54841ecce0dd47
         d600000  L0 0:41dd8000:156000 100000L/100000P F=1 B=13/13 cksum=1ffce92f48f24:ffe3e4e4b7d86aa4:ee7bb104c0e856ed:36c835667577f4d
         d700000  L0 0:41f2e000:156000 100000L/100000P F=1 B=13/13 cksum=1ffee9c991fe2:83e20cfd846b4:e30c3b0dbdf462f4:6e829a5dafc97cc1
         d800000  L0 0:42084000:156000 100000L/100000P F=1 B=13/13 cksum=1ff0463a24b15:fe33c7dbef5b541f:8f34103c8b2b895a:a707a79301a32bb8
         d900000  L0 0:421da000:156000 100000L/100000P F=1 B=13/13 cksum=20005716e7d62:24905997809e3f:7c518de6c2268611:d5900efe812d1f71
         da00000  L0 0:42330000:156000 100000L/100000P F=1 B=13/13 cksum=1ff7c3c95532f:fe4c6c78513465c6:9ce12fc2482fc977:57d6092ee70936b0
         db00000  L0 0:425dc000:156000 100000L/100000P F=1 B=13/13 cksum=200b865b32082:1c15fc2b1f5fd19:2ad2f91023a5b6f4:8e1a7ce36b2a259a
         dc00000  L0 0:42486000:156000 100000L/100000P F=1 B=13/13 cksum=201b1a0f14527:3524802524d8257:15e75b003c1e013:65fc9168f5c186ba
         dd00000  L0 0:42732000:156000 100000L/100000P F=1 B=13/13 cksum=1ff3e3c24d034:ff6519f0c2ccc904:b11005d4e382beee:218a38a53b564827
         de00000  L0 0:42888000:156000 100000L/100000P F=1 B=13/13 cksum=1ff75788be2dc:febbfe92f7ddcfd1:4f5ef773be7f7350:49aa97b615c85450
         df00000  L0 0:42b34000:156000 100000L/100000P F=1 B=13/13 cksum=1ff9426911580:ff5eccb903387f0e:46d636284983ee8a:4dd3702a03017ea5
         e000000  L0 0:429de000:156000 100000L/100000P F=1 B=13/13 cksum=2002b9f2be776:827c624cc11494:fb36b2a94d2cfa6f:7bd65f0955c09fce
         e100000  L0 0:42c8a000:156000 100000L/100000P F=1 B=13/13 cksum=1ff662e537024:fef47801166620a6:d357e5a35152c693:454f2d06b537fe3
         e200000  L0 0:42de0000:156000 100000L/100000P F=1 B=13/13 cksum=1fffd072b17c1:db19cf77331c31:eff7149e806fc1c5:4e8f768a2c9fcc33
         e300000  L0 0:42f36000:156000 100000L/100000P F=1 B=13/13 cksum=20000626dac51:fe5801e1a0ec84a9:a847b067da07567d:e705f0cd5000c01a
         e400000  L0 0:431e2000:156000 100000L/100000P F=1 B=13/13 cksum=20066b9a66a53:e7ba4749632355:fe01116e74bb7d4a:6ac6d2c805719670
         e500000  L0 0:4308c000:156000 100000L/100000P F=1 B=13/13 cksum=2000ad5256a86:fff5ae962b1a6634:c98974a1694fcffb:7ba20373549e4382
         e600000  L0 0:4348e000:156000 100000L/100000P F=1 B=13/13 cksum=1fff93beafea2:ff47e216f10996d8:119573e925ee7e9e:331a95dcf2bfd70
         e700000  L0 0:43338000:156000 100000L/100000P F=1 B=13/13 cksum=20034192746e3:a28b6064e3dfd2:6caf68e7114c9005:65aa0fdad8fdcfce
         e800000  L0 0:435e4000:156000 100000L/100000P F=1 B=13/13 cksum=1ffd9f9c83d8e:ffa2bf1cb14b77af:696f5cf6e8de97e1:20cfbcacc2d1ba69
         e900000  L0 0:4373a000:156000 100000L/100000P F=1 B=13/13 cksum=200a30c194e53:cb1f44448f9a02:16e125dc5d20635c:6c3e26a0883c9d0a
         ea00000  L0 0:439e6000:156000 100000L/100000P F=1 B=13/13 cksum=200b59f2b542e:29650ad37c3f4eb:2d59e2435bc05f3d:1bb94e513122163d
         eb00000  L0 0:43890000:156000 100000L/100000P F=1 B=13/13 cksum=200c89eb14176:1718e89bbf97ab7:beede31b49da977d:28cc21264db715cb
         ec00000  L0 0:43b3c000:156000 100000L/100000P F=1 B=13/13 cksum=1ffc62507c2f2:ff8ebda723dba50a:b718b8cf1ae224f9:7f593683b67444af
         ed00000  L0 0:43c92000:156000 100000L/100000P F=1 B=13/13 cksum=20074388609fe:ba31d1638188cc:391892f19c42acbd:265ec547dd0ee18
         ee00000  L0 0:43de8000:156000 100000L/100000P F=1 B=13/13 cksum=2003f1d01f108:f4db7e5a9d0663:23660666d8cbd0a1:49e0d5cbd2aea278
         ef00000  L0 0:43f3e000:156000 100000L/100000P F=1 B=13/13 cksum=2004e4f714970:f47b08e2751fa4:a56520ed84017267:35e84fd6b13384b9
         f000000  L0 0:44094000:156000 100000L/100000P F=1 B=13/13 cksum=200bd3038f5bb:47a0431bbe7403:a3053438c593a696:4de1549b59f8a39c
         f100000  L0 0:445fc000:156000 100000L/100000P F=1 B=14/14 cksum=1ff972690c62f:fedb30ad4c688f1f:979fe7e5e4b517e:f5bed9b1298d9db4
         f200000  L0 0:44752000:156000 100000L/100000P F=1 B=14/14 cksum=1ffa11145c081:fe5d308fa6867ca3:fe4f9331fa00b7a5:c0605c24f0b00518
         f300000  L0 0:44350000:156000 100000L/100000P F=1 B=14/14 cksum=2000b6d85bd86:8223f6aecf4d21:6eab0e471a5fd1fa:f7e6ce948e9d0a5
         f400000  L0 0:444a6000:156000 100000L/100000P F=1 B=14/14 cksum=2003de72284a9:37097003069891:6954ef28e4701ddb:40dd90148a98463a
         f500000  L0 0:441fa000:156000 100000L/100000P F=1 B=14/14 cksum=2010024d09a5a:fa52d1a3a3c5d5:a74c34e8ddbd4171:9194e0737c37473a
         f600000  L0 0:44b54000:156000 100000L/100000P F=1 B=14/14 cksum=201017e8a58ae:1f6f00b89c14f0b:ee315d54db115282:6d3126470d9abe23
         f700000  L0 0:44f56000:156000 100000L/100000P F=1 B=14/14 cksum=200253ce4e86d:ff13c540c287c9db:f790c837240f8438:1dfaea521fc9cfc
         f800000  L0 0:448a8000:156000 100000L/100000P F=1 B=14/14 cksum=20058653f8a5c:11cc68f0d65665c:530f13bb6ee8c123:468ad4758358fe1d
         f900000  L0 0:450ac000:156000 100000L/100000P F=1 B=14/14 cksum=1feb9e22dabfb:fd9de6bab66077cb:fa3e2601f75a4c49:993275dad7136efe
         fa00000  L0 0:449fe000:156000 100000L/100000P F=1 B=14/14 cksum=200f963b60f36:2ad306adc7ee378:a19eb7f99dbaaa74:b1cb036220c27023
         fb00000  L0 0:45202000:156000 100000L/100000P F=1 B=14/14 cksum=200b909c225b5:75abc61e3e39dd:7236f479312b1292:7be57b650e6f4314
         fc00000  L0 0:44caa000:156000 100000L/100000P F=1 B=14/14 cksum=20190b6ab8ab9:2e37730b3dc0105:3d3c529dcf49735:3b4693168e7e3d86
         fd00000  L0 0:454ae000:156000 100000L/100000P F=1 B=14/14 cksum=1ffe39f4c15b3:b5305892c83967:fbea5af1c8904c43:8c320e6086e3b5a2
         fe00000  L0 0:44e00000:156000 100000L/100000P F=1 B=14/14 cksum=1ff90edcf3e75:fdf380f372c458d8:fb56a8323af923f3:5dd2d455b28d7389
         ff00000  L0 0:45358000:156000 100000L/100000P F=1 B=14/14 cksum=2006fcc968062:24a2e2441378e6:d9c23691cac477a9:420c00c389cfb9d3

                segment [0000000000000000, 0000000010000000) size  256M

So there are 256*1M datablocks, I guess dsize should be 256M, not 249M.

edo1 commented 4 years ago

zdb -ddddd wrongdu 2

I revealed it works in the form zdb -ddddd wrongdu/ 2

stale[bot] commented 3 years ago

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

edo1 commented 3 years ago

Just checked with prebuilt 2.0.4 (from Proxmox VE). The same behavior. zfs list reports wrong numbers as well

root@tst:~# zfs create zpool/tst -o mountpoint=/tst -o compression=off
root@tst:~# dd if=/dev/urandom of=/tst/random bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.20773 s, 131 MB/s
root@tst:~# ls -lsh /tst/random
976M -rw-r--r-- 1 root root 1.0G Jun 15 17:02 /tst/random
root@tst:~# zfs list zpool/tst
NAME        USED  AVAIL     REFER  MOUNTPOINT
zpool/tst   976M  90.3T      976M  /tst

(zpool is 8+2 raidz2, ashift=12)

soyfrien commented 3 years ago

recordsize can't be 1M according to https://openzfs.readthedocs.io/en/latest/performance-tuning.html#dataset-recordsize

amotin commented 3 years ago

@soyfrien Old data. In openZFS recordsize can be up to 1MB by default or IIRC up to 16MB with tuning.

jixam commented 2 years ago

I noticed this as well, an 8-disk raidz2 11T pool apparently grew by 0.6T when sent to a 6-disk raidz2 pool. Modifying the script above slightly I was able to verify that my first configuration is affected while the latter is not.

disks=4 raid=raidz ashift=9 => 256M
disks=4 raid=raidz ashift=12 => 249M
disks=4 raid=raidz2 ashift=9 => 256M
disks=4 raid=raidz2 ashift=12 => 249M
disks=5 raid=raidz ashift=9 => 256M
disks=5 raid=raidz ashift=12 => 256M
disks=5 raid=raidz2 ashift=9 => 255M
disks=5 raid=raidz2 ashift=12 => 254M
disks=6 raid=raidz ashift=9 => 256M
disks=6 raid=raidz ashift=12 => 247M
disks=6 raid=raidz2 ashift=9 => 256M
disks=6 raid=raidz2 ashift=12 => 256M
disks=7 raid=raidz ashift=9 => 255M
disks=7 raid=raidz ashift=12 => 253M
disks=7 raid=raidz2 ashift=9 => 255M
disks=7 raid=raidz2 ashift=12 => 240M
disks=8 raid=raidz ashift=9 => 255M
disks=8 raid=raidz ashift=12 => 248M
disks=8 raid=raidz2 ashift=9 => 256M
disks=8 raid=raidz2 ashift=12 => 244M
stale[bot] commented 1 year ago

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.

grahamperrin commented 1 year ago

This issue has been automatically marked as "stale" …

(Let's avoid premature closure.)

stale[bot] commented 11 months ago

This issue has been automatically marked as "stale" because it has not had any activity for a while. It will be closed in 90 days if no further activity occurs. Thank you for your contributions.