search

fediverse-devnet / feditest-tests-fediverse

The tests for the fediverse testsuite
MIT License
5 stars 4 forks source link

WebFinger tests: HTTP status codes for errors #56

Closed steve-bate closed 2 weeks ago

steve-bate commented 1 month ago

Some of the tests in example-testplans/gargron-mastodon-social-imp-webfinger-server.json fail because they don't return the expected HTTP error status code (400 vs 404, for example). I'm wondering if these tests should pass on any HTTP error status code since the specific code to be expected is not specified in the RFC (unless I missed it). For example, 

    error_status = all_of(greater_than_or_equal_to(400), less_than(600))
    assert_that(response.http_status, error_status, 'Not HTTP error status')

instead of ...

    assert_that(response.http_status, equal_to(400), 'Not HTTP status 400')
steve-bate commented 1 month ago

There's some discussion related to HTTP status codes at https://stackoverflow.com/questions/3050518/what-http-status-response-code-should-i-use-if-the-request-is-missing-a-required.

From a pragmatic perspective, I believe we should have looser pass criteria for some of the error-related HTTP status codes.

One argument that may arise is that RFC 2616 is quite old and some of the related information has been updated and extended by numerous other RFCs written since then. The 422 status code (which some Fediverse servers use for a missing resource parameter) didn't exist until much later, for example.

steve-bate commented 1 month ago

After revisiting this issue, I think we should accept both 404 and 400 (and maybe 422 -- see below) as valid status codes for the case where an unsupported resource URI scheme is used. AFAICT, the RFC doesn't specify a specific status code for that case (like it does for a missing or malformed resource URI).

The older RFC 2616 definition of 400 is:

The request could not be understood by the server due to malformed syntax.

One could argue that an unsupported/unknown URI scheme does not constitute a malformed request and therefore 404 should be used. However, the more modern interpretation of 400 is:

The HyperText Transfer Protocol (HTTP) 400 Bad Request response status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (for example, malformed request syntax, invalid request message framing, or deceptive request routing). - https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400

A server developer could consider using an invalid or unsupported URI scheme to be a "client error".

Status code 422 seems like another valid option for this case. https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/422.

jernst commented 1 month ago

In today's PR #92, I have toned down some of the hard failures to soft failures, like any 4xx code is now not as bad as any code that's not a 4xx code. Does this cover it?

steve-bate commented 1 month ago

Also see comments in https://github.com/fediverse-devnet/feditest/issues/145

jernst commented 2 weeks ago

Also claiming https://github.com/fediverse-devnet/feditest/issues/166 subsumes this.