search

nachoparker / btrfs-du

Easily print BTRFS subvolume/snapshot disk usage
GNU General Public License v3.0
112 stars 17 forks source link

Difference with btrfs-size is often off by other factors than 1.024 powers #1

Closed jpluimers closed 6 years ago

jpluimers commented 6 years ago

Thanks for the script via https://github.com/agronick/btrfs-size/issues/3#issuecomment-349717664

Any obvious reasons for slight mismatches in the sizes?

I've ran the latest btrfs-size.sh and btrfs-du.sh and they seem to bee off a bit, sometimes more than just a bit.

I'd expect some 1024 based factors or 1.024 powered factors, but sometimes it's much more like for ".snapshots/632/snapshot" or ".snapshots/633/snapshot"

Any idea why?

[root@revue:~/Versioned] # ./btrfs-size/btrfs-size.sh 
==================================================================================================================================================================================================================
Snapshot / Subvolume                                               ID   Total    Exclusive Data
==================================================================================================================================================================================================================
257 gen 1165645 top level 5 path .snapshots                        257  1.41MB   1.41MB   
258 gen 1166797 top level 257 path .snapshots/1/snapshot           258  4.08GB   141.04MB 
259 gen 1165551 top level 5 path boot/grub2/i386-pc                259  2.38MB   2.38MB   
260 gen 1164831 top level 5 path boot/grub2/x86_64-efi             260  16.00KB  16.00KB  
261 gen 1165562 top level 5 path opt                               261  37.93MB  37.93MB  
262 gen 1166627 top level 5 path srv                               262  476.56MB 476.56MB 
263 gen 1166797 top level 5 path tmp                               263  2.71MB   2.71MB   
264 gen 1165644 top level 5 path usr/local                         264  260.00KB 260.00KB 
265 gen 1164831 top level 5 path var/crash                         265  16.00KB  16.00KB  
266 gen 1165551 top level 5 path var/lib/mailman                   266  32.00KB  32.00KB  
267 gen 1166720 top level 5 path var/lib/named                     267  35.88MB  35.88MB  
268 gen 1164831 top level 5 path var/lib/pgsql                     268  16.00KB  16.00KB  
269 gen 1166797 top level 5 path var/log                           269  303.68MB 303.68MB 
270 gen 1164831 top level 5 path var/opt                           270  16.00KB  16.00KB  
271 gen 1166789 top level 5 path var/spool                         271  88.00KB  88.00KB  
272 gen 1165585 top level 5 path var/tmp                           272  104.00KB 104.00KB 
797 gen 1164831 top level 5 path var/lib/machines                  797  16.00KB  16.00KB  
1148 gen 1165566 top level 257 path .snapshots/632/snapshot        1148  4.04GB   419.68MB 
1150 gen 1165566 top level 257 path .snapshots/633/snapshot        1150  4.05GB   14.94MB  
1151 gen 1165566 top level 257 path .snapshots/634/snapshot        1151  4.04GB   4.66MB   
1152 gen 1165566 top level 257 path .snapshots/635/snapshot        1152  4.05GB   16.76MB  
1153 gen 1165566 top level 257 path .snapshots/636/snapshot        1153  4.04GB   15.88MB  
1154 gen 1165563 top level 257 path .snapshots/637/snapshot        1154  4.35GB   45.39MB  
1155 gen 1165563 top level 257 path .snapshots/638/snapshot        1155  4.05GB   340.00KB 
1156 gen 1165563 top level 257 path .snapshots/639/snapshot        1156  4.05GB   672.00KB 
1189 gen 1165564 top level 257 path .snapshots/640/snapshot        1189  4.05GB   1.70MB   
1190 gen 1165563 top level 257 path .snapshots/641/snapshot        1190  4.05GB   2.31MB   
1191 gen 1165563 top level 257 path .snapshots/642/snapshot        1191  4.05GB   7.95MB   
1195 gen 1165562 top level 257 path .snapshots/643/snapshot        1195  4.38GB   62.98MB  
1212 gen 1165563 top level 257 path .snapshots/644/snapshot        1212  4.06GB   14.16MB  
1216 gen 1165564 top level 257 path .snapshots/645/snapshot        1216  4.13GB   58.97MB  
==================================================================================================================================================================================================================
                                                                Exclusive Total: 1.63GB    
[root@revue:~/Versioned] 2s # ./btrfs-du/btrfs-du 
Subvolume                                                    Total      Exclusive  ID        
─────────────────────────────────────────────────────────────────────────────────────────
.snapshots                                                   1.48MB     1.48MB     257       
.snapshots/1/snapshot                                        4.38GB     147.89MB   258       
boot/grub2/i386-pc                                           2.49MB     2.49MB     259       
boot/grub2/x86_64-efi                                        16.38kB    16.38kB    260       
opt                                                          39.78MB    39.78MB    261       
srv                                                          499.71MB   499.71MB   262       
tmp                                                          2.85MB     2.85MB     263       
usr/local                                                    266.24kB   266.24kB   264       
var/crash                                                    16.38kB    16.38kB    265       
var/lib/mailman                                              32.77kB    32.77kB    266       
var/lib/named                                                37.62MB    37.62MB    267       
var/lib/pgsql                                                16.38kB    16.38kB    268       
var/log                                                      318.46MB   318.46MB   269       
var/opt                                                      16.38kB    16.38kB    270       
var/spool                                                    90.11kB    90.11kB    271       
var/tmp                                                      106.50kB   106.50kB   272       
var/lib/machines                                             0.00B      0.00B      797       
.snapshots/632/snapshot                                      16.38kB    16.38kB    1148      
.snapshots/633/snapshot                                      4.34GB     440.07MB   1150      
.snapshots/634/snapshot                                      4.35GB     15.67MB    1151      
.snapshots/635/snapshot                                      4.34GB     4.89MB     1152      
.snapshots/636/snapshot                                      4.35GB     17.58MB    1153      
.snapshots/637/snapshot                                      4.34GB     16.65MB    1154      
.snapshots/638/snapshot                                      4.67GB     47.59MB    1155      
.snapshots/639/snapshot                                      4.35GB     348.16kB   1156      
.snapshots/640/snapshot                                      4.35GB     688.13kB   1189      
.snapshots/641/snapshot                                      4.34GB     1.78MB     1190      
.snapshots/642/snapshot                                      4.35GB     2.42MB     1191      
.snapshots/643/snapshot                                      4.35GB     8.34MB     1195      
.snapshots/644/snapshot                                      4.70GB     66.04MB    1212      
.snapshots/645/snapshot                                      4.36GB     14.84MB    1216      
─────────────────────────────────────────────────────────────────────────────────────────
                                                          Total exclusive data: 1.56GB
nachoparker commented 6 years ago

Hi,

In btrfs-du the sizes are directly reported by btrfs-tools, whereas in btrfs-size the sizes are extracted with --raw and then converted externally. This might be a source of error if the conversion is not performed perfectly.

My guess is that btrfs-du sizes are more accurate just because of this: it is directly reported by btrfs-tools with no processing.

In any case, it would be interesting to double check. Could we see the output of

sudo btrfs quota enable /
sudo btrfs qgroup show --si /

, and

sudo btrfs subvolume list /

in your system? Let's notice the difference between using --si and not using it. The difference is base 1000 vs base 1024

If you don't want quotas enabled, disable afterwards.

thanks

jpluimers commented 6 years ago

Here you go:

# btrfs quota enable /
# btrfs qgroup show --si /
qgroupid         rfer         excl 
--------         ----         ---- 
0/5           16.38kB      16.38kB 
0/257          1.90MB       1.90MB 
0/258          4.68GB     141.92MB 
0/259          2.49MB       2.49MB 
0/260         16.38kB      16.38kB 
0/261         40.20MB      40.20MB 
0/262        499.71MB     499.71MB 
0/263          2.85MB       2.85MB 
0/264        266.24kB     266.24kB 
0/265         16.38kB      16.38kB 
0/266         32.77kB      32.77kB 
0/267         37.65MB      37.65MB 
0/268         16.38kB      16.38kB 
0/269        343.04MB     343.04MB 
0/270         16.38kB      16.38kB 
0/271         90.11kB      90.11kB 
0/272         90.11kB      90.11kB 
0/274           0.00B        0.00B 
0/797         16.38kB      16.38kB 
0/1151         4.34GB     457.27MB 
0/1152         4.35GB      17.58MB 
0/1153         4.34GB      16.65MB 
0/1154         4.67GB      47.59MB 
0/1155         4.35GB     348.16kB 
0/1156         4.35GB     688.13kB 
0/1189         4.34GB       1.78MB 
0/1190         4.35GB       2.42MB 
0/1191         4.35GB       8.34MB 
0/1195         4.70GB      66.04MB 
0/1212         4.36GB      14.84MB 
0/1216         4.43GB      64.61MB 
0/1231         4.38GB       6.77MB 
0/1232         4.69GB      15.94MB 
255/274         0.00B        0.00B 
255/797       16.38kB      16.38kB 
# btrfs subvolume list /
ID 257 gen 1174947 top level 5 path .snapshots
ID 258 gen 1176351 top level 257 path .snapshots/1/snapshot
ID 259 gen 1165551 top level 5 path boot/grub2/i386-pc
ID 260 gen 1164831 top level 5 path boot/grub2/x86_64-efi
ID 261 gen 1170540 top level 5 path opt
ID 262 gen 1176336 top level 5 path srv
ID 263 gen 1176347 top level 5 path tmp
ID 264 gen 1174704 top level 5 path usr/local
ID 265 gen 1164831 top level 5 path var/crash
ID 266 gen 1165551 top level 5 path var/lib/mailman
ID 267 gen 1175896 top level 5 path var/lib/named
ID 268 gen 1164831 top level 5 path var/lib/pgsql
ID 269 gen 1176350 top level 5 path var/log
ID 270 gen 1164831 top level 5 path var/opt
ID 271 gen 1176332 top level 5 path var/spool
ID 272 gen 1172346 top level 5 path var/tmp
ID 797 gen 1164831 top level 5 path var/lib/machines
ID 1151 gen 1165566 top level 257 path .snapshots/634/snapshot
ID 1152 gen 1165566 top level 257 path .snapshots/635/snapshot
ID 1153 gen 1165566 top level 257 path .snapshots/636/snapshot
ID 1154 gen 1165563 top level 257 path .snapshots/637/snapshot
ID 1155 gen 1165563 top level 257 path .snapshots/638/snapshot
ID 1156 gen 1165563 top level 257 path .snapshots/639/snapshot
ID 1189 gen 1165564 top level 257 path .snapshots/640/snapshot
ID 1190 gen 1165563 top level 257 path .snapshots/641/snapshot
ID 1191 gen 1165563 top level 257 path .snapshots/642/snapshot
ID 1195 gen 1165562 top level 257 path .snapshots/643/snapshot
ID 1212 gen 1165563 top level 257 path .snapshots/644/snapshot
ID 1216 gen 1170499 top level 257 path .snapshots/645/snapshot
ID 1231 gen 1172343 top level 257 path .snapshots/646/snapshot
ID 1232 gen 1172345 top level 257 path .snapshots/647/snapshot
nachoparker commented 6 years ago

mmm we might have something, 

0/274           0.00B        0.00B

That qgroup doesn't correspond to an ID.

I think that this line is messing up the output. I think this happens because you have quota enabled.

Can you try

sudo btrfs quota disable /

, then reboot (just in case), and then try the commands again, see if it reappears?

Trying to think the best way to adapt to this situation

jpluimers commented 6 years ago

Hmm, now they are all empty:

# btrfs quota disable /
# reboot

wait ... then logon and sudo -i bash:

# btrfs quota enable /
# btrfs qgroup show --si /
WARNING: qgroup data inconsistent, rescan recommended
qgroupid         rfer         excl 
--------         ----         ---- 
0/5             0.00B        0.00B 
0/257           0.00B        0.00B 
0/258           0.00B        0.00B 
0/259           0.00B        0.00B 
0/260           0.00B        0.00B 
0/261           0.00B        0.00B 
0/262           0.00B        0.00B 
0/263           0.00B        0.00B 
0/264           0.00B        0.00B 
0/265           0.00B        0.00B 
0/266           0.00B        0.00B 
0/267           0.00B        0.00B 
0/268           0.00B        0.00B 
0/269           0.00B        0.00B 
0/270           0.00B        0.00B 
0/271           0.00B        0.00B 
0/272           0.00B        0.00B 
0/797           0.00B        0.00B 
0/1151          0.00B        0.00B 
0/1152          0.00B        0.00B 
0/1153          0.00B        0.00B 
0/1154          0.00B        0.00B 
0/1155          0.00B        0.00B 
0/1156          0.00B        0.00B 
0/1189          0.00B        0.00B 
0/1190          0.00B        0.00B 
0/1191          0.00B        0.00B 
0/1195          0.00B        0.00B 
0/1212          0.00B        0.00B 
0/1216          0.00B        0.00B 
0/1231          0.00B        0.00B 
0/1232          0.00B        0.00B 
0/1234          0.00B        0.00B 
0/1237          0.00B        0.00B 
# btrfs subvolume list /
ID 257 gen 1176448 top level 5 path .snapshots
ID 258 gen 1176496 top level 257 path .snapshots/1/snapshot
ID 259 gen 1176447 top level 5 path boot/grub2/i386-pc
ID 260 gen 1164831 top level 5 path boot/grub2/x86_64-efi
ID 261 gen 1176439 top level 5 path opt
ID 262 gen 1176439 top level 5 path srv
ID 263 gen 1176494 top level 5 path tmp
ID 264 gen 1176445 top level 5 path usr/local
ID 265 gen 1176439 top level 5 path var/crash
ID 266 gen 1165551 top level 5 path var/lib/mailman
ID 267 gen 1176491 top level 5 path var/lib/named
ID 268 gen 1164831 top level 5 path var/lib/pgsql
ID 269 gen 1176494 top level 5 path var/log
ID 270 gen 1176439 top level 5 path var/opt
ID 271 gen 1176490 top level 5 path var/spool
ID 272 gen 1176490 top level 5 path var/tmp
ID 797 gen 1164831 top level 5 path var/lib/machines
ID 1151 gen 1165566 top level 257 path .snapshots/634/snapshot
ID 1152 gen 1165566 top level 257 path .snapshots/635/snapshot
ID 1153 gen 1165566 top level 257 path .snapshots/636/snapshot
ID 1154 gen 1165563 top level 257 path .snapshots/637/snapshot
ID 1155 gen 1165563 top level 257 path .snapshots/638/snapshot
ID 1156 gen 1165563 top level 257 path .snapshots/639/snapshot
ID 1189 gen 1165564 top level 257 path .snapshots/640/snapshot
ID 1190 gen 1165563 top level 257 path .snapshots/641/snapshot
ID 1191 gen 1165563 top level 257 path .snapshots/642/snapshot
ID 1195 gen 1165562 top level 257 path .snapshots/643/snapshot
ID 1212 gen 1165563 top level 257 path .snapshots/644/snapshot
ID 1216 gen 1170499 top level 257 path .snapshots/645/snapshot
ID 1231 gen 1172343 top level 257 path .snapshots/646/snapshot
ID 1232 gen 1172345 top level 257 path .snapshots/647/snapshot
ID 1234 gen 1176434 top level 257 path .snapshots/648/snapshot
ID 1237 gen 1176446 top level 257 path .snapshots/649/snapshot

then after waiting a while:

# btrfs qgroup show --si /
qgroupid         rfer         excl 
--------         ----         ---- 
0/5           16.38kB      16.38kB 
0/257          2.65MB       2.65MB 
0/258          4.39GB     123.21MB 
0/259          2.48MB       2.48MB 
0/260         16.38kB      16.38kB 
0/261         40.20MB      40.20MB 
0/262        499.71MB     499.71MB 
0/263          2.85MB       2.85MB 
0/264        266.24kB     266.24kB 
0/265         16.38kB      16.38kB 
0/266         32.77kB      32.77kB 
0/267         37.65MB      37.65MB 
0/268         16.38kB      16.38kB 
0/269        357.38MB     357.38MB 
0/270         16.38kB      16.38kB 
0/271         90.11kB      90.11kB 
0/272         90.11kB      90.11kB 
0/797         16.38kB      16.38kB 
0/1151         4.34GB     457.27MB 
0/1152         4.35GB      17.58MB 
0/1153         4.34GB      16.65MB 
0/1154         4.67GB      47.59MB 
0/1155         4.35GB     348.16kB 
0/1156         4.35GB     688.13kB 
0/1189         4.34GB       1.78MB 
0/1190         4.35GB       2.42MB 
0/1191         4.35GB       8.34MB 
0/1195         4.70GB      66.04MB 
0/1212         4.36GB      14.84MB 
0/1216         4.43GB      64.61MB 
0/1231         4.38GB       6.77MB 
0/1232         4.69GB      16.48MB 
0/1234         4.68GB      16.77MB 
0/1237         5.04GB      91.36MB

Still size differences though:

# ./btrfs-du/btrfs-du /
Subvolume                                                    Total      Exclusive  ID        
─────────────────────────────────────────────────────────────────────────────────────────
.snapshots                                                   2.65MB     2.65MB     257       
.snapshots/1/snapshot                                        4.39GB     123.26MB   258       
boot/grub2/i386-pc                                           2.48MB     2.48MB     259       
boot/grub2/x86_64-efi                                        16.38kB    16.38kB    260       
opt                                                          40.20MB    40.20MB    261       
srv                                                          499.71MB   499.71MB   262       
tmp                                                          2.85MB     2.85MB     263       
usr/local                                                    266.24kB   266.24kB   264       
var/crash                                                    16.38kB    16.38kB    265       
var/lib/mailman                                              32.77kB    32.77kB    266       
var/lib/named                                                37.65MB    37.65MB    267       
var/lib/pgsql                                                16.38kB    16.38kB    268       
var/log                                                      357.39MB   357.39MB   269       
var/opt                                                      16.38kB    16.38kB    270       
var/spool                                                    90.11kB    90.11kB    271       
var/tmp                                                      90.11kB    90.11kB    272       
var/lib/machines                                             16.38kB    16.38kB    797       
.snapshots/634/snapshot                                      4.34GB     457.27MB   1151      
.snapshots/635/snapshot                                      4.35GB     17.58MB    1152      
.snapshots/636/snapshot                                      4.34GB     16.65MB    1153      
.snapshots/637/snapshot                                      4.67GB     47.59MB    1154      
.snapshots/638/snapshot                                      4.35GB     348.16kB   1155      
.snapshots/639/snapshot                                      4.35GB     688.13kB   1156      
.snapshots/640/snapshot                                      4.34GB     1.78MB     1189      
.snapshots/641/snapshot                                      4.35GB     2.42MB     1190      
.snapshots/642/snapshot                                      4.35GB     8.34MB     1191      
.snapshots/643/snapshot                                      4.70GB     66.04MB    1195      
.snapshots/644/snapshot                                      4.36GB     14.84MB    1212      
.snapshots/645/snapshot                                      4.43GB     64.61MB    1216      
.snapshots/646/snapshot                                      4.38GB     6.77MB     1231      
.snapshots/647/snapshot                                      4.69GB     16.48MB    1232      
.snapshots/648/snapshot                                      4.68GB     16.77MB    1234      
.snapshots/649/snapshot                                      5.04GB     91.41MB    1237      
─────────────────────────────────────────────────────────────────────────────────────────
                                                          Total exclusive data: 1.68GB
# ./btrfs-size/btrfs-size.sh 
==================================================================================================================================================================================================================
Snapshot / Subvolume                                               ID   Total    Exclusive Data
==================================================================================================================================================================================================================
257 gen 1176448 top level 5 path .snapshots                        257  2.52MB   2.52MB   
258 gen 1176501 top level 257 path .snapshots/1/snapshot           258  4.09GB   117.55MB 
259 gen 1176447 top level 5 path boot/grub2/i386-pc                259  2.36MB   2.36MB   
260 gen 1164831 top level 5 path boot/grub2/x86_64-efi             260  16.00KB  16.00KB  
261 gen 1176439 top level 5 path opt                               261  38.34MB  38.34MB  
262 gen 1176439 top level 5 path srv                               262  476.56MB 476.56MB 
263 gen 1176502 top level 5 path tmp                               263  2.71MB   2.71MB   
264 gen 1176501 top level 5 path usr/local                         264  260.00KB 260.00KB 
265 gen 1176439 top level 5 path var/crash                         265  16.00KB  16.00KB  
266 gen 1165551 top level 5 path var/lib/mailman                   266  32.00KB  32.00KB  
267 gen 1176491 top level 5 path var/lib/named                     267  35.91MB  35.91MB  
268 gen 1164831 top level 5 path var/lib/pgsql                     268  16.00KB  16.00KB  
269 gen 1176501 top level 5 path var/log                           269  340.84MB 340.84MB 
270 gen 1176439 top level 5 path var/opt                           270  16.00KB  16.00KB  
271 gen 1176490 top level 5 path var/spool                         271  88.00KB  88.00KB  
272 gen 1176490 top level 5 path var/tmp                           272  88.00KB  88.00KB  
797 gen 1164831 top level 5 path var/lib/machines                  797  16.00KB  16.00KB  
1151 gen 1165566 top level 257 path .snapshots/634/snapshot        1151  4.04GB   436.09MB 
1152 gen 1165566 top level 257 path .snapshots/635/snapshot        1152  4.05GB   16.76MB  
1153 gen 1165566 top level 257 path .snapshots/636/snapshot        1153  4.04GB   15.88MB  
1154 gen 1165563 top level 257 path .snapshots/637/snapshot        1154  4.35GB   45.39MB  
1155 gen 1165563 top level 257 path .snapshots/638/snapshot        1155  4.05GB   340.00KB 
1156 gen 1165563 top level 257 path .snapshots/639/snapshot        1156  4.05GB   672.00KB 
1189 gen 1165564 top level 257 path .snapshots/640/snapshot        1189  4.05GB   1.70MB   
1190 gen 1165563 top level 257 path .snapshots/641/snapshot        1190  4.05GB   2.31MB   
1191 gen 1165563 top level 257 path .snapshots/642/snapshot        1191  4.05GB   7.95MB   
1195 gen 1165562 top level 257 path .snapshots/643/snapshot        1195  4.38GB   62.98MB  
1212 gen 1165563 top level 257 path .snapshots/644/snapshot        1212  4.06GB   14.16MB  
1216 gen 1170499 top level 257 path .snapshots/645/snapshot        1216  4.13GB   61.61MB  
1231 gen 1172343 top level 257 path .snapshots/646/snapshot        1231  4.08GB   6.46MB   
1232 gen 1172345 top level 257 path .snapshots/647/snapshot        1232  4.37GB   15.72MB  
1234 gen 1176434 top level 257 path .snapshots/648/snapshot        1234  4.36GB   16.00MB  
1237 gen 1176446 top level 257 path .snapshots/649/snapshot        1237  4.69GB   87.18MB  
==================================================================================================================================================================================================================
                                                                Exclusive Total: 1.77GB

It cannot be fully explained bybtrfs-du uses the sizes in SI units, whereas btrfs-size internally uses 1024 based units but fails to show the i in the unit to reflect that (the traditional https://en.wikipedia.org/wiki/Gibibyte versus https://en.wikipedia.org/wiki/Gigabyte thing).

So I've made a comparison table with the ratios (it is likely quite a bit wider than the comment width):

# ./btrfs-size/btrfs-size.sh
========== ========== ========== ============== # ./btrfs-du/btrfs-du /
Snapshot / Subvolume ID Total Exclusive Data Subvolume Total Exclusive ID Total Exclusive Data Total Exclusive ID match Total ratio Exclusive ratio Ratio match
========== ========== ========== ============== ─────────── ─────────── ─────────── ────── ========== ============== ─────────── ───────────
257 gen 1176448 top level 5 path .snapshots 257 2.52MB 2.52MB .snapshots 2.65MB 2.65MB 257 2.52 2.52 2.65 2.65 TRUE 1.051587302 1.051587302 TRUE
258 gen 1176501 top level 257 path .snapshots/1/snapshot 258 4.09GB 117.55MB .snapshots/1/snapshot 4.39GB 123.26MB 258 4.09 117.55 4.39 123.26 TRUE 1.073349633 1.048575074 FALSE
259 gen 1176447 top level 5 path boot/grub2/i386-pc 259 2.36MB 2.36MB boot/grub2/i386-pc 2.48MB 2.48MB 259 2.36 2.36 2.48 2.48 TRUE 1.050847458 1.050847458 TRUE
260 gen 1164831 top level 5 path boot/grub2/x86_64-efi 260 16.00KB 16.00KB boot/grub2/x86_64-efi 16.38kB 16.38kB 260 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
261 gen 1176439 top level 5 path opt 261 38.34MB 38.34MB opt 40.20MB 40.20MB 261 38.34 38.34 40.2 40.2 TRUE 1.048513302 1.048513302 TRUE
262 gen 1176439 top level 5 path srv 262 476.56MB 476.56MB srv 499.71MB 499.71MB 262 476.56 476.56 499.71 499.71 TRUE 1.048577304 1.048577304 TRUE
263 gen 1176502 top level 5 path tmp 263 2.71MB 2.71MB tmp 2.85MB 2.85MB 263 2.71 2.71 2.85 2.85 TRUE 1.051660517 1.051660517 TRUE
264 gen 1176501 top level 5 path usr/local 264 260.00KB 260.00KB usr/local 266.24kB 266.24kB 264 260 260 266.24 266.24 TRUE 1.024 1.024 TRUE
265 gen 1176439 top level 5 path var/crash 265 16.00KB 16.00KB var/crash 16.38kB 16.38kB 265 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
266 gen 1165551 top level 5 path var/lib/mailman 266 32.00KB 32.00KB var/lib/mailman 32.77kB 32.77kB 266 32 32 32.77 32.77 TRUE 1.0240625 1.0240625 TRUE
267 gen 1176491 top level 5 path var/lib/named 267 35.91MB 35.91MB var/lib/named 37.65MB 37.65MB 267 35.91 35.91 37.65 37.65 TRUE 1.04845447 1.04845447 TRUE
268 gen 1164831 top level 5 path var/lib/pgsql 268 16.00KB 16.00KB var/lib/pgsql 16.38kB 16.38kB 268 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
269 gen 1176501 top level 5 path var/log 269 340.84MB 340.84MB var/log 357.39MB 357.39MB 269 340.84 340.84 357.39 357.39 TRUE 1.048556507 1.048556507 TRUE
270 gen 1176439 top level 5 path var/opt 270 16.00KB 16.00KB var/opt 16.38kB 16.38kB 270 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
271 gen 1176490 top level 5 path var/spool 271 88.00KB 88.00KB var/spool 90.11kB 90.11kB 271 88 88 90.11 90.11 TRUE 1.023977273 1.023977273 TRUE
272 gen 1176490 top level 5 path var/tmp 272 88.00KB 88.00KB var/tmp 90.11kB 90.11kB 272 88 88 90.11 90.11 TRUE 1.023977273 1.023977273 TRUE
797 gen 1164831 top level 5 path var/lib/machines 797 16.00KB 16.00KB var/lib/machines 16.38kB 16.38kB 797 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
1151 gen 1165566 top level 257 path .snapshots/634/snapshot 1151 4.04GB 436.09MB .snapshots/634/snapshot 4.34GB 457.27MB 1151 4.04 436.09 4.34 457.27 TRUE 1.074257426 1.048567956 FALSE
1152 gen 1165566 top level 257 path .snapshots/635/snapshot 1152 4.05GB 16.76MB .snapshots/635/snapshot 4.35GB 17.58MB 1152 4.05 16.76 4.35 17.58 TRUE 1.074074074 1.048926014 FALSE
1153 gen 1165566 top level 257 path .snapshots/636/snapshot 1153 4.04GB 15.88MB .snapshots/636/snapshot 4.34GB 16.65MB 1153 4.04 15.88 4.34 16.65 TRUE 1.074257426 1.048488665 FALSE
1154 gen 1165563 top level 257 path .snapshots/637/snapshot 1154 4.35GB 45.39MB .snapshots/637/snapshot 4.67GB 47.59MB 1154 4.35 45.39 4.67 47.59 TRUE 1.073563218 1.048468826 FALSE
1155 gen 1165563 top level 257 path .snapshots/638/snapshot 1155 4.05GB 340.00KB .snapshots/638/snapshot 4.35GB 348.16kB 1155 4.05 340 4.35 348.16 TRUE 1.074074074 1.024 FALSE
1156 gen 1165563 top level 257 path .snapshots/639/snapshot 1156 4.05GB 672.00KB .snapshots/639/snapshot 4.35GB 688.13kB 1156 4.05 672 4.35 688.13 TRUE 1.074074074 1.024002976 FALSE
1189 gen 1165564 top level 257 path .snapshots/640/snapshot 1189 4.05GB 1.70MB .snapshots/640/snapshot 4.34GB 1.78MB 1189 4.05 1.7 4.34 1.78 TRUE 1.071604938 1.047058824 FALSE
1190 gen 1165563 top level 257 path .snapshots/641/snapshot 1190 4.05GB 2.31MB .snapshots/641/snapshot 4.35GB 2.42MB 1190 4.05 2.31 4.35 2.42 TRUE 1.074074074 1.047619048 FALSE
1191 gen 1165563 top level 257 path .snapshots/642/snapshot 1191 4.05GB 7.95MB .snapshots/642/snapshot 4.35GB 8.34MB 1191 4.05 7.95 4.35 8.34 TRUE 1.074074074 1.049056604 FALSE
1195 gen 1165562 top level 257 path .snapshots/643/snapshot 1195 4.38GB 62.98MB .snapshots/643/snapshot 4.70GB 66.04MB 1195 4.38 62.98 4.7 66.04 TRUE 1.073059361 1.048586853 FALSE
1212 gen 1165563 top level 257 path .snapshots/644/snapshot 1212 4.06GB 14.16MB .snapshots/644/snapshot 4.36GB 14.84MB 1212 4.06 14.16 4.36 14.84 TRUE 1.073891626 1.048022599 FALSE
1216 gen 1170499 top level 257 path .snapshots/645/snapshot 1216 4.13GB 61.61MB .snapshots/645/snapshot 4.43GB 64.61MB 1216 4.13 61.61 4.43 64.61 TRUE 1.072639225 1.048693394 FALSE
1231 gen 1172343 top level 257 path .snapshots/646/snapshot 1231 4.08GB 6.46MB .snapshots/646/snapshot 4.38GB 6.77MB 1231 4.08 6.46 4.38 6.77 TRUE 1.073529412 1.047987616 FALSE
1232 gen 1172345 top level 257 path .snapshots/647/snapshot 1232 4.37GB 15.72MB .snapshots/647/snapshot 4.69GB 16.48MB 1232 4.37 15.72 4.69 16.48 TRUE 1.073226545 1.048346056 FALSE
1234 gen 1176434 top level 257 path .snapshots/648/snapshot 1234 4.36GB 16.00MB .snapshots/648/snapshot 4.68GB 16.77MB 1234 4.36 16 4.68 16.77 TRUE 1.073394495 1.048125 FALSE
1237 gen 1176446 top level 257 path .snapshots/649/snapshot 1237 4.69GB 87.18MB .snapshots/649/snapshot 5.04GB 91.41MB 1237 4.69 87.18 5.04 91.41 TRUE 1.074626866 1.048520303 FALSE
========== ========== ========== ============== ─────────── ─────────── ─────────── ────── ========== ============== ─────────── ───────────
Exclusi ve Total: 1.77GB Total exclusiv e data: 1.6 8GB
nachoparker commented 6 years ago

Thanks for the detailed info.

There are some mismatches due to the 'ghost qgroups', so I created a new version that matches better by ID. (v0.0.4)

You can re-enable quotas if you want, because it seems that the problem stemmed from that.

If there are no mismatches we can look again at the accuracy of 1024 vs 1000 base reporting.

Please, tell me if you keep seeing inconsistences, showing output of btrfs-du, qgroup show and list.

Thanks!

edit: we are specifically looking at those empty qgroups, like

0/274           0.00B        0.00B

now it should work fine even with those

jpluimers commented 6 years ago

Thanks @nachoparker below the info based on v0.0.4.

btrfs-du

# ./btrfs-du/btrfs-du 
Subvolume                                                    Total      Exclusive  ID        
─────────────────────────────────────────────────────────────────────────────────────────
.snapshots                                                   2.62MB     2.62MB     257       
.snapshots/1/snapshot                                        4.39GB     159.50MB   258       
boot/grub2/i386-pc                                           2.48MB     2.48MB     259       
boot/grub2/x86_64-efi                                        16.38kB    16.38kB    260       
opt                                                          40.20MB    40.20MB    261       
srv                                                          499.71MB   499.71MB   262       
tmp                                                          2.85MB     2.85MB     263       
usr/local                                                    266.24kB   266.24kB   264       
var/crash                                                    16.38kB    16.38kB    265       
var/lib/mailman                                              32.77kB    32.77kB    266       
var/lib/named                                                37.65MB    37.65MB    267       
var/lib/pgsql                                                16.38kB    16.38kB    268       
var/log                                                      332.82MB   332.82MB   269       
var/opt                                                      16.38kB    16.38kB    270       
var/spool                                                    90.11kB    90.11kB    271       
var/tmp                                                      90.11kB    90.11kB    272       
var/lib/machines                                             16.38kB    16.38kB    797       
.snapshots/636/snapshot                                      4.34GB     414.64MB   1153      
.snapshots/637/snapshot                                      4.67GB     58.21MB    1154      
.snapshots/640/snapshot                                      4.34GB     2.11MB     1189      
.snapshots/641/snapshot                                      4.35GB     2.42MB     1190      
.snapshots/642/snapshot                                      4.35GB     8.34MB     1191      
.snapshots/643/snapshot                                      4.70GB     66.04MB    1195      
.snapshots/644/snapshot                                      4.36GB     14.84MB    1212      
.snapshots/645/snapshot                                      4.43GB     64.61MB    1216      
.snapshots/646/snapshot                                      4.38GB     6.77MB     1231      
.snapshots/647/snapshot                                      4.69GB     16.48MB    1232      
.snapshots/648/snapshot                                      4.68GB     16.77MB    1234      
.snapshots/649/snapshot                                      5.04GB     95.89MB    1237      
─────────────────────────────────────────────────────────────────────────────────────────
                                                          Total exclusive data: 1.63GB

btrfs qgroup show /

# btrfs qgroup show /
qgroupid         rfer         excl 
--------         ----         ---- 
0/5          16.00KiB     16.00KiB 
0/257         2.50MiB      2.50MiB 
0/258         4.08GiB    152.11MiB 
0/259         2.36MiB      2.36MiB 
0/260        16.00KiB     16.00KiB 
0/261        38.34MiB     38.34MiB 
0/262       476.56MiB    476.56MiB 
0/263         2.71MiB      2.71MiB 
0/264       260.00KiB    260.00KiB 
0/265        16.00KiB     16.00KiB 
0/266        32.00KiB     32.00KiB 
0/267        35.91MiB     35.91MiB 
0/268        16.00KiB     16.00KiB 
0/269       317.40MiB    317.40MiB 
0/270        16.00KiB     16.00KiB 
0/271        88.00KiB     88.00KiB 
0/272        88.00KiB     88.00KiB 
0/797        16.00KiB     16.00KiB 
0/1153        4.04GiB    395.43MiB 
0/1154        4.35GiB     55.51MiB 
0/1189        4.05GiB      2.01MiB 
0/1190        4.05GiB      2.31MiB 
0/1191        4.05GiB      7.95MiB 
0/1195        4.38GiB     62.98MiB 
0/1212        4.06GiB     14.16MiB 
0/1216        4.13GiB     61.61MiB 
0/1231        4.08GiB      6.46MiB 
0/1232        4.37GiB     15.72MiB 
0/1234        4.36GiB     16.00MiB 
0/1237        4.69GiB     91.45MiB

btrfs subvolume list /

# btrfs subvolume list /
ID 257 gen 1176807 top level 5 path .snapshots
ID 258 gen 1178595 top level 257 path .snapshots/1/snapshot
ID 259 gen 1176447 top level 5 path boot/grub2/i386-pc
ID 260 gen 1164831 top level 5 path boot/grub2/x86_64-efi
ID 261 gen 1176439 top level 5 path opt
ID 262 gen 1178528 top level 5 path srv
ID 263 gen 1178595 top level 5 path tmp
ID 264 gen 1178593 top level 5 path usr/local
ID 265 gen 1176439 top level 5 path var/crash
ID 266 gen 1165551 top level 5 path var/lib/mailman
ID 267 gen 1178374 top level 5 path var/lib/named
ID 268 gen 1164831 top level 5 path var/lib/pgsql
ID 269 gen 1178595 top level 5 path var/log
ID 270 gen 1176439 top level 5 path var/opt
ID 271 gen 1178584 top level 5 path var/spool
ID 272 gen 1176490 top level 5 path var/tmp
ID 797 gen 1164831 top level 5 path var/lib/machines
ID 1153 gen 1165566 top level 257 path .snapshots/636/snapshot
ID 1154 gen 1165563 top level 257 path .snapshots/637/snapshot
ID 1189 gen 1165564 top level 257 path .snapshots/640/snapshot
ID 1190 gen 1165563 top level 257 path .snapshots/641/snapshot
ID 1191 gen 1165563 top level 257 path .snapshots/642/snapshot
ID 1195 gen 1165562 top level 257 path .snapshots/643/snapshot
ID 1212 gen 1165563 top level 257 path .snapshots/644/snapshot
ID 1216 gen 1170499 top level 257 path .snapshots/645/snapshot
ID 1231 gen 1172343 top level 257 path .snapshots/646/snapshot
ID 1232 gen 1172345 top level 257 path .snapshots/647/snapshot
ID 1234 gen 1176434 top level 257 path .snapshots/648/snapshot
ID 1237 gen 1176446 top level 257 path .snapshots/649/snapshot

Size comparison:

# ./btrfs-size/btrfs-size.sh
=========== =========== =========== =========== # ./btrfs-du/btrfs-du /
Snapshot / Subvolume ID Total Exclusive Data Subvolume Total Exclusive ID Total Exclusive Data Total Exclusive ID match Total ratio Exclusive ratio Ratio match
=========== =========== =========== =========== ────── ────── ────── ────── =========== =========== ────── ──────
257 gen 1176807 top level 5 path .snapshots 257 2.50MB 2.50MB .snapshots 2.62MB 2.62MB 257 2.5 2.5 2.62 2.62 TRUE 1.048 1.048 TRUE
258 gen 1178584 top level 257 path .snapshots/1/snapshot 258 4.09GB 152.10MB .snapshots/1/snapshot 4.39GB 159.49MB 258 4.09 152.1 4.39 159.49 TRUE 1.073349633 1.048586456 FALSE
259 gen 1176447 top level 5 path boot/grub2/i386-pc 259 2.36MB 2.36MB boot/grub2/i386-pc 2.48MB 2.48MB 259 2.36 2.36 2.48 2.48 TRUE 1.050847458 1.050847458 TRUE
260 gen 1164831 top level 5 path boot/grub2/x86_64-efi 260 16.00KB 16.00KB boot/grub2/x86_64-efi 16.38kB 16.38kB 260 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
261 gen 1176439 top level 5 path opt 261 38.34MB 38.34MB opt 40.20MB 40.20MB 261 38.34 38.34 40.2 40.2 TRUE 1.048513302 1.048513302 TRUE
262 gen 1178528 top level 5 path srv 262 476.56MB 476.56MB srv 499.71MB 499.71MB 262 476.56 476.56 499.71 499.71 TRUE 1.048577304 1.048577304 TRUE
263 gen 1178584 top level 5 path tmp 263 2.71MB 2.71MB tmp 2.85MB 2.85MB 263 2.71 2.71 2.85 2.85 TRUE 1.051660517 1.051660517 TRUE
264 gen 1176501 top level 5 path usr/local 264 260.00KB 260.00KB usr/local 266.24kB 266.24kB 264 260 260 266.24 266.24 TRUE 1.024 1.024 TRUE
265 gen 1176439 top level 5 path var/crash 265 16.00KB 16.00KB var/crash 16.38kB 16.38kB 265 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
266 gen 1165551 top level 5 path var/lib/mailman 266 32.00KB 32.00KB var/lib/mailman 32.77kB 32.77kB 266 32 32 32.77 32.77 TRUE 1.0240625 1.0240625 TRUE
267 gen 1178374 top level 5 path var/lib/named 267 35.91MB 35.91MB var/lib/named 37.65MB 37.65MB 267 35.91 35.91 37.65 37.65 TRUE 1.04845447 1.04845447 TRUE
268 gen 1164831 top level 5 path var/lib/pgsql 268 16.00KB 16.00KB var/lib/pgsql 16.38kB 16.38kB 268 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
269 gen 1178584 top level 5 path var/log 269 317.34MB 317.34MB var/log 332.75MB 332.75MB 269 317.34 317.34 332.75 332.75 TRUE 1.048559904 1.048559904 TRUE
270 gen 1176439 top level 5 path var/opt 270 16.00KB 16.00KB var/opt 16.38kB 16.38kB 270 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
271 gen 1178584 top level 5 path var/spool 271 88.00KB 88.00KB var/spool 90.11kB 90.11kB 271 88 88 90.11 90.11 TRUE 1.023977273 1.023977273 TRUE
272 gen 1176490 top level 5 path var/tmp 272 88.00KB 88.00KB var/tmp 90.11kB 90.11kB 272 88 88 90.11 90.11 TRUE 1.023977273 1.023977273 TRUE
797 gen 1164831 top level 5 path var/lib/machines 797 16.00KB 16.00KB var/lib/machines 16.38kB 16.38kB 797 16 16 16.38 16.38 TRUE 1.02375 1.02375 TRUE
1153 gen 1165566 top level 257 path .snapshots/636/snapshot 1153 4.04GB 395.43MB .snapshots/636/snapshot 4.34GB 414.64MB 1153 4.04 395.43 4.34 414.64 TRUE 1.074257426 1.048580027 FALSE
1154 gen 1165563 top level 257 path .snapshots/637/snapshot 1154 4.35GB 55.51MB .snapshots/637/snapshot 4.67GB 58.21MB 1154 4.35 55.51 4.67 58.21 TRUE 1.073563218 1.048639885 FALSE
1189 gen 1165564 top level 257 path .snapshots/640/snapshot 1189 4.05GB 2.01MB .snapshots/640/snapshot 4.34GB 2.11MB 1189 4.05 2.01 4.34 2.11 TRUE 1.071604938 1.049751244 FALSE
1190 gen 1165563 top level 257 path .snapshots/641/snapshot 1190 4.05GB 2.31MB .snapshots/641/snapshot 4.35GB 2.42MB 1190 4.05 2.31 4.35 2.42 TRUE 1.074074074 1.047619048 FALSE
1191 gen 1165563 top level 257 path .snapshots/642/snapshot 1191 4.05GB 7.95MB .snapshots/642/snapshot 4.35GB 8.34MB 1191 4.05 7.95 4.35 8.34 TRUE 1.074074074 1.049056604 FALSE
1195 gen 1165562 top level 257 path .snapshots/643/snapshot 1195 4.38GB 62.98MB .snapshots/643/snapshot 4.70GB 66.04MB 1195 4.38 62.98 4.7 66.04 TRUE 1.073059361 1.048586853 FALSE
1212 gen 1165563 top level 257 path .snapshots/644/snapshot 1212 4.06GB 14.16MB .snapshots/644/snapshot 4.36GB 14.84MB 1212 4.06 14.16 4.36 14.84 TRUE 1.073891626 1.048022599 FALSE
1216 gen 1170499 top level 257 path .snapshots/645/snapshot 1216 4.13GB 61.61MB .snapshots/645/snapshot 4.43GB 64.61MB 1216 4.13 61.61 4.43 64.61 TRUE 1.072639225 1.048693394 FALSE
1231 gen 1172343 top level 257 path .snapshots/646/snapshot 1231 4.08GB 6.46MB .snapshots/646/snapshot 4.38GB 6.77MB 1231 4.08 6.46 4.38 6.77 TRUE 1.073529412 1.047987616 FALSE
1232 gen 1172345 top level 257 path .snapshots/647/snapshot 1232 4.37GB 15.72MB .snapshots/647/snapshot 4.69GB 16.48MB 1232 4.37 15.72 4.69 16.48 TRUE 1.073226545 1.048346056 FALSE
1234 gen 1176434 top level 257 path .snapshots/648/snapshot 1234 4.36GB 16.00MB .snapshots/648/snapshot 4.68GB 16.77MB 1234 4.36 16 4.68 16.77 TRUE 1.073394495 1.048125 FALSE
1237 gen 1176446 top level 257 path .snapshots/649/snapshot 1237 4.69GB 91.43MB .snapshots/649/snapshot 5.04GB 95.87MB 1237 4.69 91.43 5.04 95.87 TRUE 1.074626866 1.048561741 FALSE
=========== =========== =========== =========== ────── ────── ────── ────── =========== =========== ────── ──────
Exclusiv e Total: 1.72GB Total exclusiv e data: 1.6 3GB
nachoparker commented 6 years ago

Well, looks good to me. The only thing is that there are no 'ghost qgroups', so cannot be 100% sure until we find a situation with those, but I simulated that with some fake input and I think it should be fine.

As for the sizes... I suspect btrfs-size gets some rounding errors or similar, because I just capture and print the output of btrfs qgroup show --si.

We could investigate the conversion btrfs-size performs, but I think it's safe to trust btrfs-tools.

Thoughts?

Thanks again!

jpluimers commented 6 years ago

@nachoparker I noticed at https://github.com/nachoparker/btrfs-du/blob/master/btrfs-du#L91 that it uses the non-SI unit conversion unit.

I think there are two options:

nachoparker commented 6 years ago

good catch

Well, storage normally goes in --si, so base 1000 (MB,TB...).

Unix tools tend to use --iec, so base 1024 (MiB, TiB...). For instance, ls -lh uses iec, unless you do ls -lh --si. btrfs-tools also defaults to iec.

I think that si is more appropriate, but it seems that most tools use iec, so I would use that for everything, so I would vote for the first option.

Thoughts?

jpluimers commented 6 years ago

@nachoparker I am all for consistency, so my choice is --iec.

jpluimers commented 6 years ago

I did some Googling and came up with bytesToHuman.bash in https://gist.github.com/jpluimers/0f21bf1d937fe0b9b4044f21944a90ec

It can do both IEC and SI and is pure bash.

I will integrate it with the legacy btrfs script later tonight.

jpluimers commented 6 years ago

Here you go: https://github.com/jpluimers/btrfs-du/commit/3fc64281cd6dc4470bac40b972a18368d473e5a6

Let me know if you like it and I'll submit a pull request.