Site status and eligibility test doesn't recognize my 301

[lws-team]

Checking libwebsockets.org using https://hstspreload.org/?domain=libwebsockets.org claims that there is no redirect for http. But there is

$ telnet libwebsockets.org 80
Trying 46.105.127.147...
Connected to libwebsockets.org.
Escape character is '^]'.
GET / HTTP/1.1
Host: libwebsockets.org

HTTP/1.1 301 Redirect
server: lwsws
location: https://libwebsockets.org/
content-type: text/html
content-length: 0

I have tried the location with and without the final /.

Browsers and wget / curl don't have any problem following the redirect, eg

$ wget -O- --no-hsts http://libwebsockets.org:80
--2019-09-13 09:18:18--  http://libwebsockets.org/
Resolving libwebsockets.org (libwebsockets.org)... 46.105.127.147, 2001:41d0:2:ee93::
Connecting to libwebsockets.org (libwebsockets.org)|46.105.127.147|:80... connected.
HTTP request sent, awaiting response... 301 Redirect
Location: https://libwebsockets.org/ [following]
--2019-09-13 09:18:18--  https://libwebsockets.org/
Connecting to libwebsockets.org (libwebsockets.org)|46.105.127.147|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7205 (7.0K) [text/html]

I can see your server seems to take the redirect and come back on https / http/2.0 with

    accept-encoding: = gzip
    :authority = libwebsockets.org
    :method = GET
    :path = /
    :scheme = https
    user-agent: = Go-http-client/2.0

We reply with suitable headers and send it 7K of payload before setting END_STREAM OK... it seems normal from the server end.

Could you maybe take a look at what about that feels wrong to the go http client and the script?

[lgarron]

It looks like your site is responding with a 406 depending on the user agent: package main

import (
    "fmt"
    "net/http"
)

func main() {
    resp, _ := http.Get("http://libwebsockets.org/")
    fmt.Printf("%#s", resp)
}

Result:

> go run main.go
&{406 Not Acceptable %!s(int=406) HTTP/1.0 %!s(int=1) %!s(int=0) map[Content-Length:[177] Content-Type:[text/html] Server:[lwsws]] %!s(*http.bodyEOFSignal=&{c00012c140 {0 0} false <nil> 1223340 12232d0}) %!s(int64=177) [] %!s(bool=true) %!s(bool=false) map[] %!s(*http.Request=&{GET c00011e000 HTTP/1.1 1 1 map[] <nil> <nil> 0 [] false libwebsockets.org map[] map[] <nil> map[]   <nil> <nil> <nil> <nil>}) %!s(*tls.ConnectionState=<nil>)}

hstspreload.org expects sites to send an HSTS header response regardless of user agent.

[lws-team]

Thanks... I have a blacklist of useragent strings for misbehaved bots, they get sent packing with a 406. Go-http-client is in there sure enough. Sorry for the noise!

[wlabarron]

A note in case someone else comes across this issue and is having a similar problem: I found that Cloudflare's Bot Fight Mode was intercepting the requests from hstspreload.org and causing the Error: No redirect from HTTP message.