golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.09k stars 17.68k forks source link

net/http: `(*http.Response).Cookies` have different behaviour compared with web browser #66118

Open ii64 opened 8 months ago

ii64 commented 8 months ago

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/ii64/.cache/go-build'
GOENV='/home/ii64/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/ii64/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/ii64/go'
GOPRIVATE=''
GOPROXY='https://goproxy.cn,https://gocenter.io,https://goproxy.io,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4219600399=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I was trying to get any valid cookies given in HTTP response header specifically via Set-Cookie header entry. https://go.dev/play/p/M8hiEzF_n97

What did you see happen?

In the example code, the Cookies method exclude utid because it has quotation mark chars inside cookie-value. Looking at the RFC in the comment, https://datatracker.ietf.org/doc/html/rfc6265#section-4.1.1

Quotation mark " OR in hex 0x22 is excluded, which is already expected behaviour:

 set-cookie-header = "Set-Cookie:" SP set-cookie-string
 set-cookie-string = cookie-pair *( ";" SP cookie-av )
 cookie-pair       = cookie-name "=" cookie-value
 cookie-name       = token
 cookie-value      = *cookie-octet / ( DQUOTE *cookie-octet DQUOTE )
 cookie-octet      = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
                       ; US-ASCII characters excluding CTLs,
                       ; whitespace DQUOTE, comma, semicolon,
                       ; and backslash

However, for web browsers that behaviour is totally fine.

What did you expect to see?

Interface for custom validation for valid cookies

bcmills commented 8 months ago

(attn @neild)

ii64 commented 8 months ago

Just adding an extra context here of what Chromium did, is basically allowing " and some others:

Expand ```c // IsValidCookieValue() returns whether a string matches the following // grammar: // // cookie-value = *cookie-value-octet // cookie-value-octet = %x20-3A / %x3C-7E / %x80-FF // ; octets excluding CTLs and ";" // // This can be used to determine whether cookie values contain any invalid // characters. // // Note that RFC6265bis section 4.1.1 suggests a stricter grammar for // parsing cookie values, but we choose to allow a wider range of characters // than what's allowed by that grammar (while still conforming to the // requirements of the parsing algorithm defined in section 5.2). // // For reference, see: // - https://crbug.com/238041 ```

Effectivly changing the grammar to:

cookie-value       = *cookie-value-octet
cookie-value-octet = %x20-3A / %x3C-7E / %x80-FF
                       ; octets excluding CTLs and ";"
Zxilly commented 8 months ago

https://chromium.googlesource.com/chromium/src/+/refs/heads/main/net/cookies/parsed_cookie.cc

a reference to the chromium implementation

AlexanderYastrebov commented 8 months ago

Is this a duplicate of #46443 ?

ii64 commented 8 months ago

Is this a duplicate of #46443 ?

No, this one is expecting an interface for cookie validation for cookie-value grammar customability.

alexandernst commented 3 months ago

If I understand correctly this issue, Go also doesn't support any non-ascii values in the cookie value. E.g:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    var resp http.Response

    resp.Header = map[string][]string{
        "Set-Cookie": {
            "emoji=å∫∂ƒ§πø¥†®€æ; path=/; secure",
            "text=hello world!; path=/; secure",
        },
    }

    for _, c := range resp.Cookies() {
        fmt.Printf("%+#v\n", c)
    }
}

will print only the second cookie.

NachE commented 3 months ago

I have experienced a similar problem with backslashes. Backslashes are not allowed in cookie-value but web browsers has no problem with it. You can check this behavior accessing the url httpbin-url-test which sets a cookie value using backslashes to scape unicode characters. It will be nice if a custom validator for cookie values could be implemented.

Related code: https://github.com/golang/go/blob/30b6fd60a63c738c2736e83b6a6886a032e6f269/src/net/http/cookie.go#L472