golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.9k stars 17.65k forks source link

encoding/json: tags can't have Unicode punctuation #53267

Open chipaca opened 2 years ago

chipaca commented 2 years ago

What version of Go are you using (go version)?

1.18.3

Does this issue reproduce with the latest release?

1.18.3 is the latest release at the time of writing

What operating system and processor architecture are you using (go env)?

go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/john/.cache/go-build"
GOENV="/home/john/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/john/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/john/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/john/sdk/go1.18.3"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/john/sdk/go1.18.3/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.3"
GCCGO="/usr/bin/gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1281614776=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Try to use a struct with a tag that has non-ASCII symbols in it, e.g.

type T struct {
    Foo31 string `json:"foo-3·1"`
}

https://go.dev/play/p/IwgJHZCedCE

What did you expect to see?

Tags can be anything, and JSON places no limits on this either (the name of a name/value pair is “a string”), and the encoding/json package mentions no limits, so I'd expect it to just work: the above playground link should produce

{"foo-3·1":"hi"}
<nil>

What did you see instead?

the tag is silently ignored and you get

{"Foo31":"hi"}
<nil>

this is because encoding/json's isValidTag only allows (some) ASCII symbols: https://github.com/golang/go/blob/95b68e1e02fa713719f02f6c59fb1532bd05e824/src/encoding/json/encode.go#L972-L987

seankhliao commented 2 years ago

related #39189

cc @mvdan @dsnet