Open mzky opened 3 years ago
Thanks for raising the issue.
From my read this sounds more like a feature request as opposed to a bug, correct?
@neild
The feature request is to respond to an HTTP request on an HTTPS port with a configurable error message.
Testing several major websites (www.google.com, www.amazon.com, www.microsoft.com), none of them respond with an error message to an HTTP request on port 443. Two close the connection without response, one leaves the connection open but does not respond.
I have not checked any other HTTPS server implementations (Apache, nginx, etc.) to see how they handle this condition. It would be interesting to know if any attempt to report an error to the peer in this case.
In the absence of evidence that this is a common feature, I don't think we should add this.
Hi ! This is a feature request, not a bug.
I found some examples of Nginx configuring HTTP redirection to HTTPS:https://linuxize.com/post/redirect-http-to-https-in-nginx/
I want to replace nginx's auto-jump feature with Golang. Of course, this is just a suggestion, the changes will not affect the original functions.
HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers
go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
// handle redirect here
}))
http.ListenAndServeTLS(":443", "...", "...", nil)
HTTP redirection is different from the filed issue. It's already possible by listening on both HTTP and HTTPS ports and handling the redirection within the handlers
go http.ListenAndServe(":80", http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) { // handle redirect here })) http.ListenAndServeTLS(":443", "...", "...", nil)
Yes, different port forwarding is also possible, but I need a forwarding on the same port
Go language simulates google:
They all jump to https with the same port:
An https://
URL with no port uses a default port of 443, not 80. All these are redirecting from HTTP on port 80 to HTTPS on port 443.
I would also like to make some comments on this issue
First of all, since the server can output "Client send an HTTP request to an HTTPS server", it should give the control of the output to the developer instead of burying him deeply
Secondly, to solve this problem, we should not use nginx or listen to port 80 more, which are very cumbersome
Yes, any core modifications don't seem to be standard-compliant, but, that's the truth, we need a way to intercept when "Client send an HTTP request to an HTTPS server" happens, or have a kind of friendly wrapper that allows him to Controllable output, because "Client send an HTTP request to an HTTPS server" directly output to the browser is very unfriendly, and it is uncontrollable and unexpected
I would also like that this is configurable. Maybe http.Server
could get the following field:
TLSHandshakeBadRequest func(conn net.Conn)
And if not defined, current:
io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
is called. Otherwise TLSHandshakeBadRequest(re.Conn)
is called.
Open to other field names.
i have same problem that response content not friendly, i wish can configurable the content
The nginx example you provided is incorrect, it should be like this:
# port 443
error_page 497 https://$host$request_uri;
# other ports
error_page 497 https://$host:$server_port$request_uri;
Then, attempting to access the HTTPS port using HTTP will return a 302 redirect.
Maybe we can read the "Host" request header, then return 307 status code
The following is a number of debugging screenshots, not yet implemented the above functions
src/crypto/tls/conn.go
src/net/http/server.go
test.go
I did it!
Next, I will submit a pull request.
Change https://go.dev/cl/564997 mentions this issue: net/http: configurable error message for Client sent an HTTP request …
It looks like nginx responds to an HTTP request on an HTTPS port with an error 497 (an nginx-specific code), and permits code-specific error handler overrides. The example nginx configuration from above is interesting, because it seems to indicate that nginx is parsing the full HTTP request in this case:
error_page 497 https://$host$request_uri;
This points at a question: Is the desired feature request to customize the static message we send when responding to a misdirected HTTP request, or to respond with a redirect to the correct location? The latter is substantially more complicated to implement, because it requires effectively restarting the connection.
If were were to add a feature, I could imagine something like:
type Server struct {
// HTTPOnHTTPSPortErrorResponseString is sent on an HTTPS connection
// which receives what looks like an HTTP request.
HTTPOnHTTPSPortErrorResponse string
}
But that's static, and it sounds like a common use case is a redirect. So maybe what people want is:
type Server struct {
// HTTPOnHTTPSPortErrorHandler handles HTTP requests sent to an HTTPS port.
HTTPOnHTTPSPortErrorHandler Handler
}
Although in that case, what happens when someone inevitably sets Server.Handler and Server.HTTPOnHTTPSPortErrorHandler to the same thing? Does that work? Should that work? Are there edge cases where it fails? I think implementation for this would also be complicated; unless I'm wrong, I don't think this is something we'd want to do.
A middle ground, suggested by @mitar above, is essentially a roll-your-own handler where we hand the net.Conn
to the user:
type Server struct {
TLSHandshakeBadRequest func(conn net.Conn)
}
You're going to want the already-consumed bytes from the net.Conn
as well though, no?
I don't have any clear answers here. There's been a fair bit of discussion on this issue over time, but half of it is about cases other than an HTTP request sent to an HTTPS port. (For example, redirecting from HTTP on port 80 to HTTPS on port 443, which of course is something you can do today.) Some of the example use cases (such as that nginx config) involve more than just customizing the error string.
Would a statically customizable response string be enough? That's at least implementable without adding a ton of complexity.
A static string can be redirected in the browser using JavaScript,
but it doesn't fit the 301 status code redirect that some people want
"HTTP/1.1 400 Bad Request\r\nContent-Type: text/html\r\nConnection: close\r\n\r\n<!-- Client sent an HTTP request to an HTTPS server. -->\n<script> location.protocol = 'https:' </script>\n"
HTTP/1.1 400 Bad Request
Content-Type: text/html
Connection: close
<!-- Client sent an HTTP request to an HTTPS server. -->
<script> location.protocol = 'https:' </script>
@neild Yea, I realized this in https://go-review.googlesource.com/c/go/+/564997 as well. I think we could have:
type Server struct {
TLSHandshakeBadRequest func(conn net.Conn, recordedBytes []byte)
}
But I think those recorded bytes are really optional and it might be hard for later on for implementation to change its behavior. If we look at current code, it does not use those bytes, it just writes to the connection:
io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
So I was envisioning that the TLSHandshakeBadRequest
would do the same, only maybe write a Location
header redirect.
I am not sure how much utility is in inspecting recordedBytes
?
I think it's possible to provide multiple interfaces for different use cases, like TLSConfig
's NameToCertificate
and GetCertificate
(I think the relevant configuration should be written in TLSConfig)
type Config struct {
// HttpOnHttpsPortErrorResponseString is sent on an HTTPS connection
// which receives what looks like an HTTP request.
HttpOnHttpsPortErrorResponse string
// HttpOnHttpsPortErrorRedirect is sent on an HTTPS connection
// which receives what looks like an HTTP request.
// If true, 307 redirect will be sent
HttpOnHttpsPortErrorRedirect bool
// HttpOnHttpsPortErrorHandler handles HTTP requests sent to an HTTPS port.
HttpOnHttpsPortErrorHandler func(conn net.Conn, requestBytes []byte)
Priority is from bottom to top
I am not sure how much utility is in inspecting
recordedBytes
?
307 redirect
var compiledRegExp_httpHost = regexp.MustCompile(`\r\nHost: \S+`) // "\r\nHost: local.q8p.cc:5678"
var compiledRegExp_httpPath = regexp.MustCompile(`/\S*`) // "/index.html"
func httpOnHttpsPortErrorHandler (conn net.Conn, requestBytes []byte) {
requestString := string(requestBytes)
Host := compiledRegExp_httpHost.FindString(requestString) // "\r\nHost: local.q8p.cc:5678"
if Host == "" {
io.WriteString(conn, "HTTP/1.1 400 Bad Request\r\n"+
"\r\n"+
"Missing required Host header.\n",
)
return
}
Host = Host[8:] // "local.q8p.cc:5678"
Path := compiledRegExp_httpPath.FindString(requestString) // "/index.html"
io.WriteString(conn, "HTTP/1.1 307 Temporary Redirect\r\n"+
"Location: https://"+Host+Path+"\r\n"+
"Connection: close\r\n"+
"\r\n"+
"Client sent an HTTP request to an HTTPS server.\n",
)
}
HTTP/1.1 307 Temporary Redirect
Location: https://local.q8p.cc:5678/index.html
Connection: close
Client sent an HTTP request to an HTTPS server.
I just found a more standard request parsing method
req, err := http.ReadRequest(bufio.NewReader(bytes.NewReader(requestBytes)))
if err != nil {
fmt.Println(err)
return
}
io.WriteString(conn, fmt.Sprintf(
"HTTP/1.1 307 Temporary Redirect\r\nLocation: https://%s%s\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n",
req.Host, req.URL.Path,
))
I see. To allow virtual hosts redirects. And if requestBytes
does not contain enough data (not yet all headers), you can still read more from the connection?
🤔I just tried, it doesn't work, I'll see if there's an interface to continue reading
The request content cannot exceed 576 bytes, otherwise the request cannot be parsed
I successfully read the remaining bytes from conn
server.go
// net/http: configurable error message for Client sent an HTTP request to an HTTPS server.
// https://go.dev/issue/49310
func (c *conn) httpOnHttpsPortErrorHandler(conn net.Conn, recondBytes []byte) {
// Read Response string
badRequestResponse := c.server.TLSConfig.HttpOnHttpsPortErrorResponse
if badRequestResponse == "" {
badRequestResponse = "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
}
// Handler
if handler := c.server.TLSConfig.HttpOnHttpsPortErrorHandler; handler != nil {
handler(conn, recondBytes, badRequestResponse)
return
}
// Redirect
if c.server.TLSConfig.HttpOnHttpsPortErrorRedirect {
// Read Header
req, _, err := ReadRequestForHttpOnHttpsPortErrorHandler(conn, recondBytes)
if err != nil {
io.WriteString(conn, badRequestResponse)
return
}
// Send Redirect
io.WriteString(conn, fmt.Sprintf(
"HTTP/1.1 307 Temporary Redirect\r\nLocation: https://%s%s\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n",
req.Host, req.URL.Path,
))
return
}
// Send Response string
io.WriteString(conn, badRequestResponse)
}
func ReadRequestForHttpOnHttpsPortErrorHandler(conn net.Conn, recondBytes []byte) (req *Request, reqBytes []byte, err error) {
// Max 4KB
if len(recondBytes) > 4096 {
return nil, recondBytes, errors.New("recondBytes too long")
}
// Content length may be insufficient, continue reading.
if len(recondBytes) < 4096 && !bytes.Contains(recondBytes, []byte("\r\n\r\n")) {
b := make([]byte, 4096-len(recondBytes))
// Set Timeout 1s
conn.SetReadDeadline(time.Now().Add(time.Duration(1 * time.Second)))
n, err := conn.Read(b)
if err != nil {
return nil, recondBytes, err
}
recondBytes = append(recondBytes, b[:n]...)
}
// Read Request
req, err = ReadRequest(bufio.NewReader(bytes.NewReader(recondBytes)))
if err != nil {
return nil, recondBytes, err
}
if req.Host == "" {
return nil, recondBytes, errors.New("missing required Host header")
}
return req, recondBytes, nil
}
If this is enough, I will submit a PullRequest @mitar
TLSConfig
type Config struct {
// HttpOnHttpsPortErrorResponseString is sent on an HTTPS connection
// which receives what looks like an HTTP request.
// "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
HttpOnHttpsPortErrorResponse string
// HttpOnHttpsPortErrorRedirect is sent on an HTTPS connection
// which receives what looks like an HTTP request.
// If true, 307 redirect will be sent
HttpOnHttpsPortErrorRedirect bool
// HttpOnHttpsPortErrorHandler handles HTTP requests sent to an HTTPS port.
//
// WriteString use:
// io.WriteString(conn, "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
// Parse the request header use:
// http.ReadRequestForHttpOnHttpsPortErrorHandler(conn, recondBytes)
HttpOnHttpsPortErrorHandler func(conn net.Conn, recondBytes []byte, badRequestResponse string)
@bddjr Please read contributing guide. I think this proposal has not yet been accepted. Making a pull request is fine to be able to discuss code, but until the proposal is accepted it cannot be merged.
And about your last proposal: personally, I think the Config is overkill, I think we should have only one function callback and not so many options.
Also, what is badRequestResponse
in HttpOnHttpsPortErrorHandler
?
Also, what is
badRequestResponse
inHttpOnHttpsPortErrorHandler
?
It will be TLSConfig.HttpOnHttpsPortErrorResponse
// Read Response string
badRequestResponse := c.server.TLSConfig.HttpOnHttpsPortErrorResponse
if badRequestResponse == "" {
badRequestResponse = "HTTP/1.1 400 Bad Request\r\nConnection: close\r\n\r\nClient sent an HTTP request to an HTTPS server.\n"
}
// Handler
if handler := c.server.TLSConfig.HttpOnHttpsPortErrorHandler; handler != nil {
handler(conn, recondBytes, badRequestResponse)
return
}
var srv *hlfhr.Server
func main() {
// Use hlfhr.New
srv = hlfhr.New(&http.Server{
Addr: ":5678",
Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Write something...
}),
ReadHeaderTimeout: 10 * time.Second,
IdleTimeout: 10 * time.Second,
})
// Then just use it like http.Server .
err := srv.ListenAndServeTLS("localhost.crt", "localhost.key")
fmt.Println(err)
}
✅ Customize the function in the same way as http.Server.Handler
// 307 Temporary Redirect
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hlfhr.RedirectToHttps(w, r, 307)
})
// Check Host Header
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hostname, port := hlfhr.SplitHostnamePort(r.Host)
switch hostname {
case "localhost":
//
case "www.localhost", "127.0.0.1":
r.Host = hlfhr.HostnameAppendPort("localhost", port)
default:
w.WriteHeader(421)
return
}
hlfhr.RedirectToHttps(w, r, 302)
})
// Script Redirect
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
w.WriteHeader(400)
io.WriteString(w, "<script>location.protocol='https:'</script>")
})
@mkzy The solutions I previously wrote in the proposal had bugs, so they were not officially adopted. You can now use the mod I mentioned, which will correctly read HTTP requests.
我之前在提案里写的解决方案有bug,因此官方并没有采用。您现在可以使用我提到的mod,它会正确地读取http请求。
A simple example of the redirect method.
package main
import (
"fmt"
"github.com/gin-gonic/gin"
"github.com/mzky/utils/hook"
"net/http"
)
func main() {
engine := gin.New()
engine.GET("/", func(c *gin.Context) {
c.String(http.StatusOK, "Hello World!")
})
engine.GET("/login.html", func(c *gin.Context) {
c.String(http.StatusOK, "login.html ok!")
})
// The default value of `hook` is "HTTP/1.1 302 Found\r\n\r\n<script>location.protocol='https:'</script>\r\n"
// Restore the default values of the standard library ↓
// srv.SetDefaultBadRequest("HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
srv := hook.NewServer(":1234", engine)
// custom response
//srv.SetKeepAlivesEnabled(false)
//body := "---- message ----"
//srv.SetResponse(body, func(r *http.Response) {
// r.StatusCode = 400
// r.Status = http.StatusText(400)
// r.Header.Set("Content-Type", "text/html")
//})
// custom response 2
//srv.SetRedirectPath("/login.html") // Optional redirection address
//srv.SetResponse("", func(r *http.Response) {
// r.StatusCode = 307
// r.Status = http.StatusText(307)
// r.Header.Set("Timing-Allow-Origin", "*")
// r.Header.Set("Non-Authoritative-Reason", "HSTS") // HSTS mode to redirect faster than `script`
// r.Header.Set("x-xss-protection", "0")
// r.Header.Set("Cross-Origin-Resource-Policy", "Cross-Origin")
// r.Header.Set("Content-Type", "text/html; charset=utf-8")
// r.Header.Set("Location", "https://192.168.0.188:7569") // Optional redirection address
//})
fmt.Println(srv.ListenAndServeTLS("server.pem", "server.key"))
}
@mzky 你代码写得一团糟。
你正在使用的方案与hlfhr弃用的旧版本相似,这会导致网址与请求头超过576字节时没被完整读取,最终导致客户端显示connection was reset。(你应该知道我写那么多代码不是没有原因的)
你是通过劫持Write实现,如果未来标准库支持了这个功能,可能会导致你的方案不能正常工作。
我不清楚使用unsafe获取tls的私有函数会不会导致go版本1.23无法正常使用。
http响应内容还有更轻量的,在确保请求头全部读取完成的情况下,可以直接返回这个
"HTTP/1.1 400 Bad Request\r\nConnection: close\r\nCotent-Type: text/html\r\n\r\n<script>location.protocol='https:'</script>"
In theory, implementing this feature only requires an additional buffer and handler parameter. When the first byte of the request is detected to be like an HTTP method, the TLS handshake is skipped and the HTTP/1.1 handler is directly entered.
@mzky 你代码写得一团糟。
你正在使用的方案与hlfhr弃用的旧版本相似,这会导致网址与请求头超过576字节时没被完整读取,最终导致客户端显示connection was reset。(你应该知道我写那么多代码不是没有原因的)
你是通过劫持Write实现,如果未来标准库支持了这个功能,可能会导致你的方案不能正常工作。
我不清楚使用unsafe获取tls的私有函数会不会导致go版本1.23无法正常使用。
http响应内容还有更轻量的,在确保请求头全部读取完成的情况下,可以直接返回这个
"HTTP/1.1 302 Found\r\nConnection: close\r\n\r\n<script>location.protocol='https:'</script>"
After verification in multiple browsers, they can meet my needs.
Test the 7439 kB request header, and the verification result is normal:
@mzky 你在这个issue里除了提出问题以外,没有给出有用的内容。直到我发现这个提案之前,你都还在扯nginx监听80端口跳转443端口这种错误的例子。
https://github.com/bddjr/hlfhr
It uses the http.ReadRequest
to read the request headers, and its custom Handler uses the http.Handler
format, which handles HTTP requests in a standard way that is close to the standard, rather than giving a bunch of characters for the user to deal with.
The custom Handler for this mod does not support Hijack
and Keep-Alive.
srv.HttpOnHttpsPortErrorHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
url := "https://" + r.Host + r.URL.Path
if r.URL.ForceQuery || r.URL.RawQuery != "" {
url += "?" + r.URL.RawQuery
}
w.Header().Set("Location", url)
w.WriteHeader(302)
})
The existing solution can solve this problem without using the standard library, so it has been closed.
Hi ! This is a feature request, not a bug.
I found some examples of Nginx configuring HTTP redirection to HTTPS:https://linuxize.com/post/redirect-http-to-https-in-nginx/
- Old version Parameters: ··· rewrite ^(.*)$ https://$host$1 permanent; ···
- New version Parameters: ··· return 301 https://$server_name$request_uri; ···
I want to replace nginx's auto-jump feature with Golang. Of course, this is just a suggestion, the changes will not affect the original functions.
nginx监听80端口跳转443端口
nginx监听80端口跳转443端口不是我说的,麻烦看清楚先
你自己看看你说了什么
Hi ! This is a feature request, not a bug. I found some examples of Nginx configuring HTTP redirection to HTTPS:https://linuxize.com/post/redirect-http-to-https-in-nginx/
- Old version Parameters: ··· rewrite ^(.*)$ https://$host$1 permanent; ···
- New version Parameters: ··· return 301 https://$server_name$request_uri; ···
I want to replace nginx's auto-jump feature with Golang. Of course, this is just a suggestion, the changes will not affect the original functions.
nginx监听80端口跳转443端口
nginx监听80端口跳转443端口不是我说的,麻烦看清楚先
我回复的是这位老哥,并且80转443也不是我说的,没有必要在这里讨论这些issue以外的事情
neild的意思是他不知道反向代理器有没有这个功能,结果你举的80端口的例子,而不是 error_page 497 ,直到我发言才纠正过来
The nginx example you provided is incorrect, it should be like this:
# port 443 error_page 497 https://$host$request_uri; # other ports error_page 497 https://$host:$server_port$request_uri;
Then, attempting to access the HTTPS port using HTTP will return a 302 redirect. For example: http://q8p.cc:443
我佩服您的能力,能把 HTTP 重定向理解成 HTML 重定向,能把用于 HTTP 端口的重定向写法放到 HTTPS 端口,能在已经有很完善的mod的情况下自己再用别人已经弃用的不稳定的方案造一遍
Please reopen the issue, it is not yet solved in standard library.
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
yes.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
Start the HTTPS service written in golang, and the user accesses the HTTP address.
What did you expect to see?
Display localized language information or customize html page or automatically jump to https address.
What did you see instead?
Chrome and Firefox: display the following information
Other browsers: The IE browser prompts a 400 error, and the instructional content is not displayed. Browsers using the Chromium kernel do not display any visible content. Users will not be able to understand the current situation.