influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.19k stars 3.51k forks source link

Slice bounds out of range in Influx OSS 1 on Windows Server #25028

Open snadschlaeger-faw opened 1 month ago

snadschlaeger-faw commented 1 month ago

We are currently experiencing a problem with Influx OSS 1.8.10 on Windows Server. After some time, Influx completely stops working with "slice bounds out of range" error. It looks like a int overflow in mmap_windows.go, creating a slice for opening index files.

Steps to reproduce: Because the actual reason for the error is unknown, it is not easy to reproduce. In our case, the error occurs after the database reaches around 250GB of data and a cardinality around 121 Mio. + , using tsi1.

Expected behaviour: Normal Influx behaviour.

Actual behaviour: The Influx database stops working. A "slice bounds out of range" error (see log) prevents it from working, or even starting again.

Environment info:

Config: No special config.

Logs: The log (even though everything is turned on), only contains this relevant information:

`panic: runtime error: slice bounds out of range [:1073741893] with length 1073741824

goroutine 1 [running]: github.com/influxdata/influxdb/pkg/mmap.Map(0xc0008eda40, 0x3d, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /go/src/github.com/influxdata/influxdb/pkg/mmap/mmap_windows.go:45 +0x486 github.com/influxdata/influxdb/tsdb.(SeriesIndex).Open.func1(0xc00015a000, 0xa0, 0x16575a0) /go/src/github.com/influxdata/influxdb/tsdb/series_index.go:69 +0xb6 github.com/influxdata/influxdb/tsdb.(SeriesIndex).Open(0xc00015a000, 0xc00015a000, 0x2) /go/src/github.com/influxdata/influxdb/tsdb/series_index.go:84 +0x36 github.com/influxdata/influxdb/tsdb.(SeriesPartition).Open.func1(0xc000c4c2c0, 0x37, 0x1ff) /go/src/github.com/influxdata/influxdb/tsdb/series_partition.go:88 +0x12b github.com/influxdata/influxdb/tsdb.(SeriesPartition).Open(0xc000c4c2c0, 0xc0008c6280, 0x1) /go/src/github.com/influxdata/influxdb/tsdb/series_partition.go:95 +0xae github.com/influxdata/influxdb/tsdb.(SeriesFile).Open(0xc000160b40, 0x0, 0x0) /go/src/github.com/influxdata/influxdb/tsdb/series_file.go:91 +0x36e github.com/influxdata/influxdb/tsdb.(Store).openSeriesFile(0xc000033000, 0xc000233ec0, 0x6, 0xc0000357d0, 0x2c, 0xc000096f60) /go/src/github.com/influxdata/influxdb/tsdb/store.go:516 +0x140 github.com/influxdata/influxdb/tsdb.(Store).loadShards(0xc000033000, 0x0, 0x0) /go/src/github.com/influxdata/influxdb/tsdb/store.go:315 +0x14df github.com/influxdata/influxdb/tsdb.(Store).Open(0xc000033000, 0x0, 0x0) /go/src/github.com/influxdata/influxdb/tsdb/store.go:221 +0x290 github.com/influxdata/influxdb/cmd/influxd/run.(Server).Open(0xc00038c280, 0xc000541d40, 0xc00038c280) /go/src/github.com/influxdata/influxdb/cmd/influxd/run/server.go:444 +0xde4 github.com/influxdata/influxdb/cmd/influxd/run.(Command).Run(0xc000223ef0, 0xc000092090, 0x2, 0x3, 0x0, 0xc000092090) /go/src/github.com/influxdata/influxdb/cmd/influxd/run/command.go:149 +0xde7 main.(*Main).Run(0xc000387f20, 0xc000092090, 0x2, 0x3, 0xc000387f20, 0x4411c1) /go/src/github.com/influxdata/influxdb/cmd/influxd/main.go:81 +0x14e main.main() /go/src/github.com/influxdata/influxdb/cmd/influxd/main.go:45 +0x144`