Open StefanBruens opened 6 years ago
The defrag in kernel will check if the extent size is smaller than the desired size (which is 32MiB as set by the plugin) so the reflink breaking should not happen for the large extents.
$> /usr/sbin/filefrag -v /usr/lib/sysimage/rpm/Packages
Filesystem type is: 9123683e
File size of /usr/lib/sysimage/rpm/Packages is 119873536 (29266 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 0: 41431285.. 41431285: 1: shared
1: 1.. 1: 41431284.. 41431284: 1: 41431286: shared
2: 2.. 2005: 41418336.. 41420339: 2004: 41431285: shared
3: 2006.. 3869: 41485147.. 41487010: 1864: 41420340: shared
4: 3870.. 5660: 42789245.. 42791035: 1791: 41487011: shared
5: 5661.. 7351: 43204908.. 43206598: 1691: 42791036: shared
6: 7352.. 7827: 47230750.. 47231225: 476: 43206599: shared
7: 7828.. 7828: 47202769.. 47202769: 1: 47231226: shared
8: 7829.. 8957: 47231227.. 47232355: 1129: 47202770: shared
9: 8958.. 10529: 47431169.. 47432740: 1572: 47232356: shared
10: 10530.. 11910: 48537624.. 48539004: 1381: 47432741: shared
11: 11911.. 12945: 48681080.. 48682114: 1035: 48539005: shared
12: 12946.. 14499: 48885660.. 48887213: 1554: 48682115: shared
13: 14500.. 16045: 41145954.. 41147499: 1546: 48887214: shared
14: 16046.. 16046: 41134733.. 41134733: 1: 41147500: shared
15: 16047.. 16053: 41147501.. 41147507: 7: 41134734: shared
16: 16057.. 17063: 41427289.. 41428295: 1007: 41147511: shared
17: 17064.. 18062: 41595470.. 41596468: 999: 41428296: shared
18: 18063.. 19060: 41601520.. 41602517: 998: 41596469: shared
19: 19061.. 20019: 42776324.. 42777282: 959: 41602518: shared
20: 20020.. 20939: 42868357.. 42869276: 920: 42777283: shared
21: 20940.. 21855: 43950672.. 43951587: 916: 42869277: shared
22: 21856.. 22752: 44058986.. 44059882: 897: 43951588: shared
23: 22753.. 24496: 41403185.. 41404928: 1744: 44059883: shared
24: 24497.. 25368: 44114935.. 44115806: 872: 41404929: shared
25: 25369.. 26237: 44153880.. 44154748: 869: 44115807: shared
26: 26238.. 27749: 47353469.. 47354980: 1512: 44154749: shared
27: 27750.. 28505: 47336920.. 47337675: 756: 47354981: shared
28: 28506.. 29261: 47384779.. 47385534: 756: 47337676: shared
29: 29262.. 29265: 47365639.. 47365642: 4: 47385535: last,shared,eof
/usr/lib/sysimage/rpm/Packages: 30 extents found
$> sudo btrfs filesystem defragment -v -f -r -t $((64*1024*1024)) /usr/lib/sysimage/rpm/Packages
$> /usr/sbin/filefrag -v /usr/lib/sysimage/rpm/Packages Filesystem type is: 9123683e
File size of /usr/lib/sysimage/rpm/Packages is 119873536 (29266 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1002: 47803913.. 47804915: 1003: shared
1: 1003.. 2001: 48325808.. 48326806: 999: 47804916: shared
2: 2002.. 2999: 48514434.. 48515431: 998: 48326807: shared
3: 3000.. 3958: 48682115.. 48683073: 959: 48515432: shared
4: 3959.. 4878: 48685174.. 48686093: 920: 48683074: shared
5: 4879.. 5746: 48732345.. 48733212: 868: 48686094: shared
6: 5747.. 6600: 48750367.. 48751220: 854: 48733213: shared
7: 6601.. 7438: 48826925.. 48827762: 838: 48751221: shared
8: 7439.. 8226: 48961438.. 48962225: 788: 48827763: shared
9: 8227.. 8951: 49060434.. 49061158: 725: 48962226: shared
10: 8952.. 9668: 12734017.. 12734733: 717: 49061159: shared
11: 9669.. 10375: 15088029.. 15088735: 707: 12734734: shared
12: 10376.. 11059: 15090851.. 15091534: 684: 15088736: shared
13: 11060.. 11725: 15470301.. 15470966: 666: 15091535: shared
14: 11726.. 13049: 37246799.. 37248122: 1324: 15470967: shared
15: 13050.. 13711: 37296116.. 37296777: 662: 37248123: shared
16: 13712.. 14882: 37327447.. 37328617: 1171: 37296778: shared
17: 14883.. 16053: 37371038.. 37372208: 1171: 37328618: shared
18: 16057.. 16715: 44053314.. 44053972: 659: 37372212: shared
19: 16716.. 17371: 46584234.. 46584889: 656: 44053973: shared
20: 17372.. 18017: 46725185.. 46725830: 646: 46584890: shared
21: 18018.. 18655: 47581014.. 47581651: 638: 46725831: shared
22: 18656.. 19289: 47685701.. 47686334: 634: 47581652: shared
23: 19290.. 20552: 39101005.. 39102267: 1263: 47686335: shared
24: 20553.. 21183: 39267296.. 39267926: 631: 39102268: shared
25: 21184.. 22438: 36158632.. 36159886: 1255: 39267927: shared
26: 22439.. 23065: 37296778.. 37297404: 627: 36159887: shared
27: 23066.. 24319: 41290100.. 41291353: 1254: 37297405: shared
28: 24320.. 24946: 41350627.. 41351253: 627: 41291354: shared
29: 24947.. 25572: 41432985.. 41433610: 626: 41351254: shared
30: 25573.. 26823: 41213704.. 41214954: 1251: 41433611: shared
31: 26824.. 27433: 41438670.. 41439279: 610: 41214955: shared
32: 27434.. 28349: 16016876.. 16017791: 916: 41439280:
33: 28350.. 29265: 31404890.. 31405805: 916: 16017792: last,eof
/usr/lib/sysimage/rpm/Packages: 34 extents found
So the target size of 64M (I was mistaken about 32M, that was default of defrag
itself) is too much on an average aged filesystem and leads to overwriting the whole file and breaking the reflinks. At mininimum we can reduce it to 4M. An extent layout analysis or smarter defrag would be better in the long run.
I've checked one of my rolling distro installations where the system gets updated frequently and the rpm database is usually fragmented. There were way more fragments of all sizes. The post-update defragmentation was able to suqeeze the whole file to larger blocks. So I think the fragmentation itself is not enough to determine if the defragmentation should be done or not. Changing the target extent size to 32M could produce a similar result so I don't think this would make the situation worse in the above case.
I'm going to set the target size to 32MiB and keep the issue open as the extent analysis is a better solution.
The defragmentation is run unconditionally, even if the majority of extens is still large, e.g.: ` /usr/sbin/filefrag -v /usr/lib/sysimage/rpm/Packages
This is after installing a single package, with deactivated defrag plugin
Defragmentation causes high disk usage (man btrfs-filesystem):