data to filter

[mottosso]

It looks like data of ls(pattern, data) is a filter on the paths discovered via the pattern.

I'd vouch for calling it filter, but that's a reserved Python keyword, so how about query?

ls(pattern, query={"key": "value"})

At the moment, data is a bit ambiguous I think, it sounds like something it being added.

[BigRoy]

Somehow the term data got linked in my mind to being a dictionary so that is why I chose it. Preferrably I would end up doing something like:

ls(pattern, 
   include={"key": "value"}, 
   exclude={"other_key": "other_value"})

What do you think?

[mottosso]

I think ls is really good as-is, minimal and very flexible. I would leave it that way, and implement new features elsewhere.

I'm not sure about those two extra arguments, I can see the use, but it smells complicated.

[BigRoy]

Actually only one is new. Because include would be the same as the query (replacing it since it might be more descriptive; or it would be query and exclude).

Originally I was thinking also allowing this:

ls(pattern, include={"item": ("ben", "john")}

This would query allowing both to ben and john, similarly exclude would also allow it. An example would be querying all published contents that are of family model or review:

ls(pattern, include={"family": ("model", "review")}

Or the exact opposite. All of the published contents that are not of those families:

ls(pattern, exclude={"family": ("model", "review")}

[mottosso]

This would query allowing both to ben and john

I think it's getting too complicated.

I'd separate getting and filtering; getting is the only part where the filesystem needs to be involved, filtering doesn't need to be.

That will simplify testing as well, as filtering can be tested without a filesystem. Baking the two together complicates everything IMO.

[BigRoy]

Not entirely.

The concept is that a filter optimizes the query (by a lot) since it's a top-down search unless for static parts. So if we would exclude the Ben as an item then any of its contents are skipped and the file system doesn't have to be touched for it.

Otherwise the data (or query or include) argument could be removed as well. But that's specifically the thing that allows us to do a targeted query.

[mottosso]

Ah, that makes sense.

In that case, I'm not sure any more. I can see this thing getting very complicated, when at he moment it's very simple.

[BigRoy]

Ok.

I've given this some thought. For me data is fine, but I can see that there's better terminology that is likely more appropriate.

Unfortunately query doesn't cut it. In reality the pattern is what defines the query. And the data only filters it down to an inclusive set of those key/values instead of the being the full query.

So... I think include is a best match for what it does. It defines an inclusive search within the pattern for the given key/values.

[BigRoy]

Forgot to reference the issue in the commit. Refactored it in this commit: https://github.com/BigRoy/pather/commit/8ffb9f54b095fb546eddcd89dd420a1efc6d325c