MUST NOT -> SHOULD NOT return status code

[mnot]

Clients MAY interpret HTTP 400, 404, or 405 response codes as indications that the URI template is not correct. Servers MUST NOT return these response codes if the request is well-formed and the URI matches a supported template.

Generally, we shy away from saying "server MUST NOT return x status code " in HTTP because there are situations where that's the best thing to do.

For example, a MASQUE server could decide that sending a 404 is the best thing to do to hide the existence of a service from a given client.

[DavidSchinazi]

That paragraph was added as part of #107, and thinking about it more it adds more confusion that clarity so I'm going to remove the paragraph in #163.

[bemasc]

As the author of this paragraph, I'm concerned about its removal.

The purpose of this text is to enable clients to figure out that the template they are using is not correct. This is especially important given the default template, which will almost always be incorrect.

Without a rule like this, error codes are specific to a single URI (i.e. a single destination), and the client never knows if MASQUE might work for a different destination. It therefore has to try MASQUE for every destination (even though it will always fail if the template is wrong), or implement and tune a heuristic to abandon the use of MASQUE after failures with many destinations (which creates compatibility problems with MASQUE servers that serve limited destinations).

[DavidSchinazi]

I'm assuming that by "the template is not correct" you mean "the client was configured with a different URI template than what the UDP proxy expects", please correct me if that's not the case.

Since these error codes don't tell the client what URI template they should have used, the clients will not treat any errors in in a different way, so specifying error codes doesn't change any outcome here. This is very much a binary outcome, either it works or it doesn't. Clients will implement heuristics based on this binary response, not on error codes. As Mark points out, error codes can't be used for this purpose in HTTP.

[bemasc]

This is very much a binary outcome, either it works or it doesn't.

No, HTTP error codes can distinguish between transient failures (e.g. 429), permanent/cacheable failures (e.g. 404), and even non-error results that some clients might not support (e.g. 307/8). The "Proxy-Status" header provides further detail, and is normatively recommended in this draft precisely because a binary response is insufficient.

Clients will implement heuristics based on this binary response, not on error codes.

Some clients might indeed do that, but HTTP also has many minimal or nearly-stateless clients for whom heuristics of this kind would be difficult to implement well. Also, the draft provides no mention of any such heuristics, or guidance on how they ought to work.

As Mark points out, error codes can't be used for this purpose in HTTP.

Mark suggested a SHOULD, which does not seem to be the same as "can't be used for this purpose".

[DavidSchinazi]

What is the purpose of the MAY and SHOULD in the resulting paragraph? In what way would it improve interoperability?

[bemasc]

My goal with this text was to enable clients to determine whether an HTTP CONNECT server also supports MASQUE. I'm not too picky about how that works, but there ought to be some way to do it. The "MAY" documents an approved way to do that, and implicitly threatens servers with incompatibility if they don't hold up their end. Whether we say "SHOULD" or "MUST" on the server side, the "MAY" for the client creates an effective normative requirement on servers.

To be clear, my main concern here has been that server implementors will be tempted to represent NXDOMAIN or "No Route To Host" as 404, etc. That's a very logical mapping that would break a lot of client heuristics, but is not prohibited by anything in the draft.

[DavidSchinazi]

I share that goal. The problem is that this gets tricky because in HTTP a server and an origin aren't the same thing. So my thought is that our best outcome is a two-pronged approach:

1. keep CONNECT-UDP simple, browsers will re-use their "if QUIC breaks often, mark QUIC as broken and use TCP" approach
2. publish something more formalized like draft-schwartz-masque-access-descriptions later to have something that solves the problem for more than just CONNECT-UDP

I think the underlying realization here is that browsers will need to interop with servers that do the wrong thing via a simple "mark CONNECT-UDP as broken for this proxy" heuristic, and that once they do that, adding requirements on error codes doesn't gain anything that isn't already covered by Proxy-Status

[bemasc]

I think that's reasonably likely to be true for browsers, but not all clients are browsers.

Part of the problem is that the TCP fallback heuristic is more complicated than that. The client needs to fall back to TCP, see that TCP works, and only then attribute the failure as likely proxy-related. This is only possible if the proxy client is also the HTTP client, able to change the HTTP version. It does not work for system-level proxy layers, as they cannot control, or even track, UDP->TCP fallback.

Ultimately, some heuristic somewhere will probably make it work, but we're creating waste and delay that could be avoided by giving some clear instructions about how to tell if this thing is a MASQUE proxy.

[DavidSchinazi]

I agree, though I'm not aware of system proxy layers having plans to implement CONNECT-UDP just yet. By the time those happen we'll have a more formalized approach. We can't avoid waste and delay with SHOULDs and MAYs so might as well wait for the formalized solution to tackle that problem.

[mnot]

I suggested a SHOULD because saying that you MUST send a particular status code in a particular situation precludes sending other status codes that might be more relevant. It also encourages clients to equate seeing that status code to only that error condition, even when it might have been generated by another component.

However, there's something else going on here --

Without a rule like this, error codes are specific to a single URI (i.e. a single destination), and the client never knows if MASQUE might work for a different destination.

If you want to say something about many resources, it's best to do so explicitly. While you can infer things about server state from a few HTTP status codes (e.g., 505), most are specific to the resource in question. Saying otherwise is changing the semantics of those status codes, and that isn't something this specification can do.