Provide a library for use by other programs

[rdoeffinger]

A single program is better than nothing, but it can't replace a whole ecosystem of tools. It would be nice to have a library that other programs can call, preferably just providing a compression-aware stat/lstat. Even better would be a library that can be LD_PRELOADED and replaces lstat/stat (or for most Linux systems rather __lxstat seems to be what is called) overriding only the block count. This would then show compression naturally in programs like ncdu without needing any code changes in them. While not entirely risk-free, most of the concerns of replacing st_blocks with the actual allocation shouldn't apply for this usage.

[kilobyte]

With current restrictions, I don't think it makes sense to make compsize a library.

The most cumbersome part is requiring root. A library that's usable only by root-only programs is not that nice, and LD_PRELOAD plays badly with sudo.

Then, if you're interested about space saved, it's inter-file sharing that gives the most. For that, even compsize itself is hardly adequate — I pondered writing something akin to ncdu but so far, I don't have enough tuits.

Compression data should be properly exposed kernel side first.

[rdoeffinger]

Not that I disagree much, but... Didn't think of the sudo aspect, though for my use-case it seems easy enough to just start a shell first with sudo. I wouldn't think the inter-file sharing is relevant to my use-case (standard desktop install), and even if it was you'd have a massive visualization issue anyway. And having it properly exposed by the kernel sure would be best, but how many years have we been waiting for that? Will that even happen in the next 10 years? Right now basically anyone who needs to save space needs to run e.g. ncdu and in parallel try to dig into the details with compsize. Even with all the restrictions, ANYTHING would be a vast improvement in the current situation. At least that's my point of view on it.

[namibj]

I can only add my idea of how to provide this: add it to (a fork of) ncdu, with a limited way that allows the user to provide a relative and absolute limit of how much of a file needs to be referenced from another (think mp3 files with changed ID3 tags) to treat them similar to hardlinks (maybe adding the unique part of the clone when seeing the size of it, or maybe just letting it fall under the table).

Even if you'd have to adapt ncdu to drop it's partial recomputation, and pretty much just use it's display/navigation layer, that'd be awesome.