proposal: encoding/json: add error var to compare the returned error when using json.Decoder.DisallowUnknownFields()

[jaswdr]

What version of Go are you using (go version)?

$ go version

go version devel +ffc7bc55f3 Tue Oct 9 10:35:08 2018 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jaswdr/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jaswdr"
GOPROXY=""
GORACE=""
GOROOT="/home/jaswdr/go"
GOTMPDIR=""
GOTOOLDIR="/home/jaswdr/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build749620831=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
    "log"
    "bytes"
    "encoding/json"
)

type T struct {
    Bar string `json:"bar"`
}

func main() {
    str := []byte(`{"foo":"bar"}`)
    obj := new(T)

    dec := json.NewDecoder(bytes.NewReader(str))
    dec.DisallowUnknownFields()
    err := dec.Decode(&obj)
    if err != nil { // Whant to check if the specific unkown fields error happen
        log.Fatal(err)
    }

    log.Println(obj)
}

What did you expect to see?

Some way to check if the error happen.

...
if err == json.UnknownFieldsError {
 ...
}
...

What did you see instead?

No way to do this instead of checking the error string

---

If this is accepted I really want to work on it.

[gopherbot]

Change https://golang.org/cl/152297 mentions this issue: encoding/json: created json.UnknownFieldsError

[andybons]

Per discussion with @golang/proposal-review, putting on hold until we can review JSON issues en masse.

[marco-m]

Hello @andybons, do you have any idea of when you will be able to review these JSON issues en masse? Thanks.

[s3rj1k]

Looks absolutely ugly in retrospect to golang 1.13 with new error handling (errors.Is).

[jaswdr]

@s3rj1k please be more objective, this was the best solution in the time when I face this problem, instead of judge, please provide an example using errors.Is, so I can update this PR

[s3rj1k]

@jaswdr I was talking about golang core library mess. not your solution :)

[jaswdr]

@s3rj1k ack

[MissingRoberto]

Can we revisit this? In my case, I am trying to detect if clients are sending JSON requests with unknown fields. I don't want to make the request fail, but I want to make it visible by printing some warnings.

Being able to handle the error properly would allow me to achieve that.

[markus-azer]

the only solution i could think about without writing lot of code is

                 var p *entity.Product
        dec := json.NewDecoder(r.Body)
        dec.DisallowUnknownFields() //WARNNING return only one unknown field

        err := dec.Decode(&p)

        switch {
        case err == io.EOF:
            w.WriteHeader(http.StatusBadRequest)
            w.Write([]byte("Provide valid Body"))
            return
        case err != nil && strings.Contains(err.Error(), "json: unknown field"): // if the error string changed within the package then the below case will work `Provide valid JSON`
            m := regexp.MustCompile(`\".+\"`)
            field := m.FindString(err.Error())
            w.WriteHeader(http.StatusBadRequest)
            w.Write([]byte(field + " : Not Allowed"))
            return
        case err != nil:
            w.WriteHeader(http.StatusBadRequest)
            w.Write([]byte("Provide valid JSON"))
            return
        }

[ghost]

It would be great to see this get added. As @markus-azer explained, the workaround for now would be to compare to the error message itself. Instead of using strings.Contains, I went with strings.HasPrefix.

if strings.HasPrefix(err.Error(), "json: unknown field") {
    // ...
}

This issue is similar to https://github.com/golang/go/issues/30715, which was recently resolved by introducing an error, so message comparison is no longer needed.

Looking forward to this one being resolved too.

[scottjmaddox]

Also worth noting that it does not appear to be possible to return the full json path to the unexpected field, as is. In contrast, json.UnmarshalTypeError includes the full path to the offending field.

[cha0ran]

I never expected that this issue would be shelved for as long as six years. This is truly unbelievable. Meanwhile, the pull request to fix this issue was also shelved due to an error in the key data type.

[Delta456]

Are there any plans on still making this an error? Would be useful!

[ianlancetaylor]

json changes are mostly focused on the v2 package. See #63397. CC @dsnet @mvdan