search

json-iterator / go

A high-performance 100% compatible drop-in replacement of "encoding/json"
http://jsoniter.com/migrate-from-go-std.html
MIT License
13.42k stars 1.03k forks source link

fatal error: concurrent map writes on new type of map initialization #618

Open un000 opened 2 years ago

un000 commented 2 years ago

Hello, got an issue after deployment. My service restarted with a following panic:

fatal error: concurrent map writes

goroutine 26280 [running]:
runtime.throw({0x1284e53?, 0x7ff5ce6ddc48?})
\t/usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc12c758088 sp=0xc12c758058 pc=0x4362b1
runtime.mapassign(0x10ac900?, 0xc139ab65d0?, 0x8?)
\t/usr/local/go/src/runtime/map.go:595 +0x4d6 fp=0xc12c758108 sp=0xc12c758088 pc=0x40f3b6
github.com/json-iterator/go.createEncoderOfType(0xc139ab6660, {0x146c570, 0xc0003923f0})
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:244 +0x9f fp=0xc12c758148 sp=0xc12c758108 pc=0x97a59f
github.com/json-iterator/go.encoderOfType(0xc139ab6660, {0x146c570, 0xc0003923f0})
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:227 +0x45 fp=0xc12c7581a0 sp=0xc12c758148 pc=0x97a3a5
github.com/json-iterator/go.encoderOfMapKey(0xc139ab6660, {0x146c570?, 0xc0003923f0})
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:133 +0x350 fp=0xc12c758218 sp=0xc12c7581a0 pc=0x980b50
github.com/json-iterator/go.(*dynamicMapKeyEncoder).Encode(0xc1615cad98, 0xc153ca5bc8?, 0xc12c7582b0?)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:237 +0x72 fp=0xc12c758250 sp=0xc12c758218 pc=0x981472
github.com/json-iterator/go.(*mapEncoder).Encode(0xc139ab6690, 0xc153ca5bc8, 0xc100ae0180)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:263 +0x3c9 fp=0xc12c7582f8 sp=0xc12c758250 pc=0x981929
github.com/json-iterator/go.(*onePtrEncoder).Encode(0xc13537e8a0, 0xc19fac49f0, 0xc12c758370?)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:219 +0x82 fp=0xc12c758330 sp=0xc12c7582f8 pc=0x97a322
github.com/json-iterator/go.(*Stream).WriteVal(0xc100ae0180, {0x10b1d60, 0xc19fac49f0})
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:98 +0x158 fp=0xc12c7583a0 sp=0xc12c758330 pc=0x979638
github.com/json-iterator/go.(*dynamicEncoder).Encode(0xc100ae0180?, 0xc19facc584?, 0x3?)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_dynamic.go:15 +0x39 fp=0xc12c7583d0 sp=0xc12c7583a0 pc=0x97bcf9
github.com/json-iterator/go.(*placeholderEncoder).Encode(0xc00955ad80?, 0xc19fad22d0?, 0x40d4e5?)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:332 +0x22 fp=0xc12c7583f8 sp=0xc12c7583d0 pc=0x97ad62
github.com/json-iterator/go.(*mapEncoder).Encode(0xc139b94960, 0xc19fad22d0, 0xc100ae0180)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:269 +0x243 fp=0xc12c7584a0 sp=0xc12c7583f8 pc=0x9817a3
github.com/json-iterator/go.(*structFieldEncoder).Encode(0xc139b94ae0, 0xfa3a5d?, 0xc100ae0180)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_struct_encoder.go:110 +0x56 fp=0xc12c758518 sp=0xc12c7584a0 pc=0x98f6f6
github.com/json-iterator/go.(*structEncoder).Encode(0xc139b94bd0, 0x0?, 0xc100ae0180)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_struct_encoder.go:158 +0x765 fp=0xc12c758600 sp=0xc12c758518 pc=0x990105
github.com/json-iterator/go.(*OptionalEncoder).Encode(0xc0003b4000?, 0x0?, 0x0?)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_optional.go:70 +0xa4 fp=0xc12c758650 sp=0xc12c758600 pc=0x9872c4
github.com/json-iterator/go.(*onePtrEncoder).Encode(0xc13bf7aef0, 0xc19fad22c0, 0xc19fac4b10?)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:219 +0x82 fp=0xc12c758688 sp=0xc12c758650 pc=0x97a322
github.com/json-iterator/go.(*Stream).WriteVal(0xc100ae0180, {0x10949c0, 0xc19fad22c0})
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:98 +0x158 fp=0xc12c7586f8 sp=0xc12c758688 pc=0x979638
github.com/json-iterator/go.(*frozenConfig).MarshalToString(0xc0003b4000, {0x10949c0, 0xc19fad22c0})
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/config.go:289 +0xa6 fp=0xc12c758770 sp=0xc12c7586f8 pc=0x970546
github.com/json-iterator/go.MarshalToString(...)
\t/go/pkg/mod/github.com/json-iterator/go@v1.1.12/adapter.go:44
...

The code

import  "github.com/aerospike/aerospike-client-go/v5"
func MarshalJSON(r *aerospike.Record) string {
    res, _ := jsoniter.MarshalToString(r)
    return res
}

Env

golang:1.18.0
github.com/json-iterator/go v1.1.12
un000 commented 2 years ago

Looks like I found a reproducer(run it multiple times with -race to see):

package tracer_test

import (
    "sync"
    "testing"

    jsoniter "github.com/json-iterator/go"
)

func MarshalJSON(i any) (string, error) {
    res, err := jsoniter.MarshalToString(i)
    return res, err
}

func TestMarshalJSON(t *testing.T) {
    m1 := map[int]any{1: 5, 5:44}
    m2 := map[string]any{"a": "b", "c": "9"}
    m3 := map[any]string{1: "6", "7": "8"}

    wg := sync.WaitGroup{}
    for i := 0; i < 1000; i++ {
        wg.Add(3)
        go func() {
            defer wg.Done()
            MarshalJSON(m1)
        }()
        go func() {
            defer wg.Done()
            MarshalJSON(m2)
        }()
        go func() {
            defer wg.Done()
            MarshalJSON(m3)
        }()
    }
    wg.Wait()
}
==================
WARNING: DATA RACE
Read at 0x00c00005a0e0 by goroutine 13:
  github.com/json-iterator/go.(*placeholderEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:332 +0x38
  github.com/json-iterator/go.(*numericMapKeyEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:222 +0x126
  github.com/json-iterator/go.(*dynamicMapKeyEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:237 +0x116
  github.com/json-iterator/go.(*mapEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:263 +0x861
  github.com/json-iterator/go.(*onePtrEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:219 +0xa3
  github.com/json-iterator/go.(*Stream).WriteVal()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:98 +0x1f7
  github.com/json-iterator/go.(*frozenConfig).MarshalToString()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/config.go:289 +0xe6
  github.com/json-iterator/go.MarshalToString()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/adapter.go:44 +0x59

Previous write at 0x00c00005a0e0 by goroutine 10:
  github.com/json-iterator/go.createEncoderOfType()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:246 +0x198
  github.com/json-iterator/go.encoderOfType()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:227 +0x66
  github.com/json-iterator/go.encoderOfMapKey()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:133 +0x633
  github.com/json-iterator/go.(*dynamicMapKeyEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:237 +0xbb
  github.com/json-iterator/go.(*mapEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect_map.go:263 +0x861
  github.com/json-iterator/go.(*onePtrEncoder).Encode()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:219 +0xa3
  github.com/json-iterator/go.(*Stream).WriteVal()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/reflect.go:98 +0x1f7
  github.com/json-iterator/go.(*frozenConfig).MarshalToString()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/config.go:289 +0xe6
  github.com/json-iterator/go.MarshalToString()
      /Users/un0/go/pkg/mod/github.com/json-iterator/go@v1.1.12/adapter.go:44 +0x59
1298509345 commented 2 years ago

same error

khanh0123 commented 2 years ago

I have same error Version go: v1.15 Json-iterator v1.1.12

zwczou commented 2 years ago

go 1.18 json-iterator v1.1.12 same error 一样的错误

bighw commented 2 years ago

六月份的问题，现在还没有解决吗？不准备解决了吗？