Freaky / Compactor

A user interface for Windows 10 filesystem compression
MIT License
1.18k stars 49 forks source link

Prune excluded directories #8

Closed wefalltomorrow closed 4 years ago

wefalltomorrow commented 5 years ago

I have the following folders excluded for all drives:

:\Windows* :\$Recycle.Bin* :\Documents and Settings* :\System Volume Information*

Yet Compactor stills goes through each file in said folders whilst adding them to the total 'excluded' value, would it not be more efficient to not scan any excluded directories and subdirectories?

Dr-Emann commented 5 years ago

This would be really easy to do, where we say:

let walker = WalkDir::new(&self.path)
            .into_iter()
            ....
            .enumerate();

We just have to add a line saying:

let walker = WalkDir::new(&self.path)
            .into_iter()
            .filter_entry(|e| excludes.is_match(e.path()))
            ....
            .enumerate();

The only downside is that the number and size of excluded files will be inaccurate.

Freaky commented 5 years ago

I'd probably keep scanning excluded files in order to minimise the inaccuracy. Not much point skipping them - it doesn't make the directory scan much cheaper, just saves checking the compressed size.

It's notable that currently compressed files do not get checked against the excludes list - they go into the compressed pile, so you can still decompress them.

agret commented 4 years ago

I'd probably keep scanning excluded files in order to minimise the inaccuracy. Not much point skipping them - it doesn't make the directory scan much cheaper, just saves checking the compressed size.

If you are excluding the Windows folder for example it has many many subfolders with many files in each of those. Seems like a total waste of time to be crawling that when it's on the excluded list. There are many other cases where you might have a directory tree with tons of sub files that you don't want to waste time processing also.

Freaky commented 4 years ago

To be clear, if you've excluded a folder, it and all its contents are skipped entirely. If you've excluded a pattern of files inside a folder, like, say C:\bla\*.log, matching files will still have their compressed file size checked like any other file so it'll still give you an accurate assessment of the size of the folder.

I really should publish a new release with this in, shouldn't I :)

Freaky commented 4 years ago

Excluded subdirectories now pruned entirely in v0.8.0.