ByteArena / backends

Byte Arena - Digital playground for Autonomous agents
https://get.bytearena.com
1 stars 0 forks source link

[viz] REDIS buffer full #178

Open xtuc opened 6 years ago

xtuc commented 6 years ago

Service did crash, logs:

root@sd-58059:~# docker-system logs viz                                                                                                                                                                                                       
+ DOCKER_HOST=unix:///data/docker-system/docker.sock
+ docker logs viz
+ exec /usr/bin/viz-server --port 8071 --mqhost redis.net.bytearena.com --apiurl https://graphql.net.bytearena.com --record-dir /var/log/records
{"time":"2017-09-15T09:14:52Z","service":"arena-server","message":"Byte Arena Viz Server v0.1; serving assets from /usr/bin/webclient/","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"mq","message":"Subscribed to bus viz.message","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"mq","message":"Subscribed to bus game.stopped","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"viz-server","message":"VIZ Listening on :8071","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"viz-server","message":"Serving a new game Sandbox ARENA with 1 contestants (GAMEID=1, TPS=20)","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"viz-server","message":"Serving a new game Sandbox ARENA SQUARE with 2 contestants (GAMEID=2, TPS=20)","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"viz-server","message":"Serving a new game Demo ARENA with 4 contestants (GAMEID=3, TPS=20)","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"MutliArenaRecorder","message":"wrote record metadata for game facfbd73-ed29-4342-bce0-714e18bc5243","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:52Z","service":"viz:message","message":"received batch of 20 message(s) for arena server facfbd73-ed29-4342-bce0-714e18bc5243","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:53Z","service":"MutliArenaRecorder","message":"wrote record metadata for game 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:53Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:53Z","service":"viz:message","message":"received batch of 20 message(s) for arena server facfbd73-ed29-4342-bce0-714e18bc5243","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:56Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:56Z","service":"viz:message","message":"received batch of 20 message(s) for arena server facfbd73-ed29-4342-bce0-714e18bc5243","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:57Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:14:57Z","service":"viz:message","message":"received batch of 20 message(s) for arena server facfbd73-ed29-4342-bce0-714e18bc5243","context":{"hostname":"5cd90575bc79"}}
[...]
{"time":"2017-09-15T09:16:21Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:16:21Z","service":"MutliArenaRecorder","message":"stopped recording for arena facfbd73-ed29-4342-bce0-714e18bc5243","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:16:22Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:16:23Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:16:24Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:16:25Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
{"time":"2017-09-15T09:16:26Z","service":"viz:message","message":"received batch of 20 message(s) for arena server 008477df-e822-4e59-a282-2309c04ddf43","context":{"hostname":"5cd90575bc79"}}
panic: bufio: buffer full

goroutine 14 [running]:
github.com/bytearena/bytearena/common/mq.(*Client).Subscribe.func1(0xc42041dc00, 0xc4203b7100)
        /go/src/github.com/bytearena/bytearena/common/mq/client.go:161 +0x1c1
created by github.com/bytearena/bytearena/common/mq.(*Client).Subscribe
        /go/src/github.com/bytearena/bytearena/common/mq/client.go:175 +0x235
xtuc commented 6 years ago

Reported upstream https://github.com/go-redis/redis/issues/641

netgusto commented 6 years ago

@xtuc Use a fork