doc: Unclear about multiple path_regexp matchers

[rawtaz]

On https://caddyserver.com/docs/caddyfile/matchers#path-regexp it says that:

• the syntax is path_regexp [<name>] <regexp>
• "The main advantage of specifying a name is if more than one regexp matcher is used in the same named matcher"

The not matcher is listed under "Standard matchers" at https://caddyserver.com/docs/caddyfile/matchers, hence it is presumably to fully be considered a matcher and not something else. It should then follow that if you write the following configuration snippet:

@forbidden not {
    path_regexp php ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php$
    path_regexp folders1 ^/(config|tmp|core|lang)
    path_regexp folders2 ^/(libs|vendor|plugins|misc|node_modules)
    path_regexp ht /\.ht
}

.. you have written "more than one regexp matcher" (of the type path_regexp) that is "used in the same named matcher" (of the type not, named "forbidden"), and that this should be fully valid since you have supplied unique names for each path_regexp matcher.

But this doesn't work, with Caddy 2.8.4 you get the following error: Error: adapting config using caddyfile: regular expression can only be used once per named matcher.

Is the documentation wrong? I don't see how I am misunderstanding it, if that is what I'm doing. It would be great if someone could improve the docs and the section about naming the path_regexp matcher, so that it's clearer how to properly do "more than one regexp matcher [is] used in the same named matcher".

EDIT: I'm fully aware that one can concatenate those four regexes into one, but that is besides the point. The discussion here is the documentation.

[francislavoie]

It does say at the bottom of path_regexp:

There can only be one path_regexp pattern per named matcher.

I'm not sure where the confusion comes from. I don't really follow your logic.

You can use an expression matcher to have more than one path_regexp, that fact is also mentioned in the docs.

To explain the technical reasons, the mechanism that allows multiple of the same matchers is whether they can be merged into one. For most matchers like path, they can be merged cause they're implemented as a list of strings as input. But for path_regexp, the MatchPathRE implementation was written as a single struct which holds a name + pattern (and compiling the regexp for maximum performance at runtime), and therefore cannot be merged (there's nowhere to store the multiple patterns in this structure). It is an unfortunate limitation based on how the matcher was originally designed. But yeah there's ways around it, i.e. expression.

[rawtaz]

Thanks for your reply!

I am with you on the fact that one can use expression to combine multiple path_regexp, but that one has drawbacks explained in other issues on this same topic (e.g. that parts of the regex needs escaping and it all becomes a bit unwieldy for that and other reasons). Regardless, it doesn't address the documentation which is what this issue is about and I think is a bit contradictory.

So let's answer your question, what the confusion is. If we read https://caddyserver.com/docs/caddyfile/matchers#path-regexp we see two sentences:

• The main advantage of specifying a name is if more than one regexp matcher is used in the same named matcher.
• _There can only be one pathregexp pattern per named matcher.

I'm suggesting that these two sentences somewhat contradict each other. The first one suggests that by naming your path_regexp (which I have four of in my example earlier) you can use more than one regexp matcher in the same named matcher (which the not matcher in my example is one of). The second one suggests you can not use more than one regexp matcher (of which the path_regexp presumably is one) per named matcher.

So the first sentence suggests that my example should be valid, but the error message and second sentence suggest it's not. Does this make it clearer what I'm talking about? Am I misunderstanding something?

[francislavoie]

Oh, it means if you have header_regexp and/or vars_regexp in the same named matcher as a path_regexp, then you'd want names for each so they don't clobber eachother's outputs. I guess we can clarify that.

[rawtaz]

I see. So what is allow more than one of in the same named matcher is different types of regex matchers, but not more than one of a single type of regex matcher. That makes sense. Thanks for the answer :)

[francislavoie]

I will say though, there is this degenerate case that would allow multiple of the same:

@forbidden not {
    not not path_regexp php ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php$
    not not path_regexp folders1 ^/(config|tmp|core|lang)
    not not path_regexp folders2 ^/(libs|vendor|plugins|misc|node_modules)
    not not path_regexp ht /\.ht
}

That's because each not matcher itself has a MatcherSet, so no merging is attempted. But that's obviously kinda insane.

[rawtaz]

Can't we then add a matcher named really that is the same as not not, so that we can do:

@forbidden not {
    really path_regexp php ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php$
    really path_regexp folders1 ^/(config|tmp|core|lang)
    really path_regexp folders2 ^/(libs|vendor|plugins|misc|node_modules)
    really path_regexp ht /\.ht
}

That would make it cleaner :-)

[francislavoie]

lmao I guess you could make a plugin with that if you want, pretty trivial implementation. I rather come up with a proper solution, that's obviously a hack.

[rawtaz]

It was a joke, believe it or not :D

[francislavoie]

Clarified here: https://github.com/caddyserver/website/commit/59a848c26d6a9c8f76053e5efdd5244457cdfceb