Expand the Globber to be able to match individual files or documents against pattern(s)

[Perksey]

When migrating Silk.NET I found myself needing to filter documents based on patterns in a parallel module. Currently there's no easy way to do this.

Use case:

• in the input files, there are JSON files (not standard metadata ones) and regular HTML documents
• the JSON files are ran through 2 modules to ultimately generate some ObjectDocuments from them
• the ObjectDocuments are then flown through a third module which inspects the ObjectDocuments, and adds them as metadata to the appropriate regular documents (receiving both types of documents as input)

Possible Solutions:

• add an overload of the Globber methods which take an enumerable of IFiles instead of a IDirectory
• add an overload which uses strings or documents
• something else?

Thanks for building a great framework!

[daveaglick]

Can I just say I love you you've jumped right in and are already doing cool out-of-the-box stuff :)

There may be something already existing that can help here. Given an IEnumerable<TDocument>, there are a couple extension methods called FilterSources and FilterDestinations that might help. Both extension methods take one or more globbing patterns and apply it to the document enumerable, filtering for matching source and destination paths respectively. Does that seem like it's on the right track, or have a I totally missed something here?

[Perksey]

Thanks! We actually ended up getting this working (as you saw on Twitter) by just coalescing the pipelines, you can have an explore here: https://github.com/dotnet/Silk.NET/tree/feature/heavy-wip-website/src/Website

It's a great framework and saved me making my own generator from scratch (which I was actually prepared to do and already had a prototype working)!

Can't speak highly enough about your work with Statiq, truly is something else :D

[Perksey]

Will close this as the Filter* modules are a good choice and is what we've moved to use now, I don't think expanding the separate Globber API is particularly necessary.