Rewrite manifest include/exclude support

[gsnedders]

Currently, wpt run's --include and --exclude are processed within wptrunner by wptrunner.manifestinclude (via wptrunner.testloader.TestFilter); this essentially implements inclusion/exclusion based on a trie where the longest prefix wins (i.e., if you exclude "/" and include "/foobar/", "/foobar/*" is included as a longer prefix).

--include and --exclude are a bit weird insofar as they sorta support paths and sorta support URLs. Essentially, I believe the goal here is to support scheme-less and host-less URLs and path(-ish) things.

I think what we want to do is convert the string to an absolute (filesystem) path and see if that's within the manifest's tests_root, otherwise treat it as a URL.

Then:

• If it's a path in the manifest, match all tests generated from that path.
  • If it's not a path (because we're either generated on the server, like foo.any.html, or contain query parameters), iterate through all items in the directory and match against them.

Note that we currently support wildcards when the arguments are filesystem paths; we should at least support that for the final path element (given some directories have thousands of tests in them, especially in CSS), but is it worthwhile supporting them in the entire path (though it complicates walking the trie), for example so you can do webrtc*?

The ultimate goal here is to integrate the include/exclude support closer with the manifest and therefore avoid having to create all the items before filtering in the common case.

[jgraham]

I think this sounds sensible. Generic wildards seem like they come at substantial cost and the only use case for generic wildcards not expanded by the shell would be where there is a URL/path mismatch like webrtc/*.worker.any.html. Which seems possibly useful but probably requires a slow path working like today's path where we construct everything in order to match against it.

[gsnedders]

By "generic wildcards" do you mean anywhere in the path, or even on the final path component? Because there's still a substantial cost regardless of how we implement wildcards, though I do think they're useful.

[gsnedders]

This should now be much more achievable with the new manifest version.

From my memory of looking into this, given the level at which the filtering is applied, wptrunner.testloader applies it over a group of manifests (rather than just one), so we probably want to move some degree of support of "manifest sets" or something into manifest, and then have the ability to do the filtering both there and of individual manifests.

[gsnedders]

https://github.com/web-platform-tests/wpt/compare/manifest-include-exclude is a very early start here, though that branch also contains a very early version of the current manifest version which is outdated. The SkipNode class may be of some help here, though.