search

influxdata / kapacitor

Open source framework for processing, monitoring, and alerting on time series data
MIT License
2.32k stars 492 forks source link

calcThroughput crashing? #1671

Open omere2 opened 7 years ago

omere2 commented 7 years ago

bug 100% reproducible, latest git code. While running batch live replay, server is crashing with this message:

Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: github.com/influxdata/kapacitor.(*node).start.func1(0xc42037e000, 0x0, 0x0, 0x0)
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/node.go:181 +0x8e
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: created by github.com/influxdata/kapacitor.(*node).start
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/node.go:159 +0x5d
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: goroutine 5137 [select]:
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: github.com/influxdata/kapacitor.(*QueryNode).runBatch(0xc420884f20, 0x0, 0x0, 0x0, 0xc4205
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/batch.go:372 +0x282
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: github.com/influxdata/kapacitor.(*QueryNode).(github.com/influxdata/kapacitor.runBatch)-fm
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/batch.go:151 +0x48
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: github.com/influxdata/kapacitor.(*node).start.func1(0xc420884f20, 0x0, 0x0, 0x0)
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/node.go:181 +0x8e
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: created by github.com/influxdata/kapacitor.(*node).start
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/node.go:159 +0x5d
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: goroutine 5156 [select]:
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: github.com/influxdata/kapacitor.(*ExecutingTask).calcThroughput(0xc4204bcbd0)
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/task.go:428 +0x2c8
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: created by github.com/influxdata/kapacitor.(*ExecutingTask).start
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]:         /home/ubuntu/go/src/github.com/influxdata/kapacitor/task.go:231 +0x155
Nov 12 18:14:05 ip-172-31-21-11 kapacitord[20240]: goroutine 5187 [select]:
Nov 12 18:14:06 ip-172-31-21-11 systemd[1]: kapacitor.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Nov 12 18:14:06 ip-172-31-21-11 systemd[1]: kapacitor.service: Unit entered failed state.
Nov 12 18:14:06 ip-172-31-21-11 systemd[1]: kapacitor.service: Failed with result 'exit-code'.
Nov 12 18:14:06 ip-172-31-21-11 systemd[1]: kapacitor.service: Service hold-off time over, scheduling restart.
Nov 12 18:14:06 ip-172-31-21-11 systemd[1]: Stopped Time series data processing engine..
Nov 12 18:14:06 ip-172-31-21-11 systemd[1]: Started Time series data processing engine..
omere2 commented 7 years ago

probably related to out of memory of #1672