`lib.fileset.fileFilter` should allow filtering by file subpath/components

[infinisil]

Issue description

Currently the predicate of lib.fileset.fileFilter only gets name, type (and soon hasExt).

There's no way to filter by the subpath/components of the file, something like this:

lib.fileset.fileFilter (file:
  lib.lists.hasSuffix [ "foo" "bar" ] file.components
) ./.

(hasSuffix currently also doesn't exist)

Some ideas:

• file.components :: [ String ]: File components, always at least one element for the file's name
• file.subpath :: String: The file's subpath
• file.dirComponents :: [ String ]: Directory components, can be empty for files in the root level
• file.dirSubpath :: String: Directory subpath, can be ./. for files in the root level

This feature request is the motivation behind https://github.com/NixOS/nixpkgs/pull/267384.

This issue is sponsored by Antithesis :sparkles:

[roberth]

This would be the most powerful function, and users catch on to that. If the function can do anything, so can they. However, their sets will require more computation at evaluation time, and there's a good chance they'll be harder to read.

Meanwhile, I haven't seen a concrete use case that needs such a function. Do you have a real world motivating use case for this? ("parity with filterSource" would be one or a few degrees short of real world.)

[infinisil]

Indeed, I can't think of a good motivation for this right now. In fact I can only think of a reason why we shouldn't do this:

People might try this:

lib.fileset.fileFilter (file:
  head file.components == ".git"
) ./.

But that's really bad for performance.

[infinisil]

I do think we need something else for cleanSource parity. I think it should be something like lib.fileset.directoryFilter, which allows recursion to stop early. I'll think about it a bit and open another issue. (btw @roberth do you want a ping for these issues?)

[roberth]

An alternative solution that is equally powerful is for the users to do some readDir and returning file sets from that. Might not feel "integrated", but seems quite efficient, because it's constructive rather than O(nm) steps where m is the complexity of the predicate reaching its return value.

[roberth]

Not sure about pings tbh :sweat_smile:. I feel like it's a nicely rounded library now, so I'm just say a lot of "do we really need this" and how about just this + that. There's also quite a bit of value in keeping a library small. Ping me if you need me to explain why something isn't needed I guess!

[infinisil]

Opened https://github.com/NixOS/nixpkgs/issues/269517 just to track the potential of a cleanSource replacement

[infinisil]

@roberth Here's a more use-case oriented feature that's needed, relating to this one: https://github.com/NixOS/nixpkgs/issues/271307