search

nspcc-dev / neofs-s3-gw

NeoFS S3 Protocol Gateway
GNU Affero General Public License v3.0
9 stars 15 forks source link

Panic under high load #1032

Open EESergey opened 5 days ago

EESergey commented 5 days ago

This problem is reproduced in 100% of cases, with 50 PUT threads per node (12core, 24GB RAM).

roman-khimov commented 4 days ago

We need the panic output here.

roman-khimov commented 10 hours ago 
panic: runtime error: slice bounds out of range [:8192] with capacity 4096

goroutine 1300612 [running]:
bufio.(*Reader).ReadSlice(0xc004d97020, 0xa)
    bufio/bufio.go:351 +0x22e
github.com/nspcc-dev/neofs-s3-gw/api/auth/signer/v4.readChunkLine(0xc00212de90?)
    github.com/nspcc-dev/neofs-s3-gw/api/auth/signer/v4/chunked_reader.go:191 +0x1c
github.com/nspcc-dev/neofs-s3-gw/api/auth/signer/v4.(*chunkedReader).beginChunk(0xc004d97080)
    github.com/nspcc-dev/neofs-s3-gw/api/auth/signer/v4/chunked_reader.go:67 +0x25
github.com/nspcc-dev/neofs-s3-gw/api/auth/signer/v4.(*chunkedReader).Read(0xc004d97080, {0xc00964c000?, 0xff5d60?, 0x1?})
    github.com/nspcc-dev/neofs-s3-gw/api/auth/signer/v4/chunked_reader.go:153 +0x134
github.com/nspcc-dev/neofs-s3-gw/api/metrics.(*readCounter).Read(0xc00aea0378, {0xc00964c000?, 0xc006a3c2c0?, 0x0?})
    github.com/nspcc-dev/neofs-s3-gw/api/metrics/api.go:226 +0x28
github.com/nspcc-dev/neofs-s3-gw/api/layer.wrapReader.func1()
    github.com/nspcc-dev/neofs-s3-gw/api/layer/object.go:904 +0x75
created by github.com/nspcc-dev/neofs-s3-gw/api/layer.wrapReader in goroutine 1293869
    github.com/nspcc-dev/neofs-s3-gw/api/layer/object.go:901 +0x145