search

linkedin / Burrow

Kafka Consumer Lag Checking
Apache License 2.0
3.73k stars 797 forks source link

Out of memory #549

Open Jacuo opened 5 years ago

Jacuo commented 5 years ago

I try to run newest burrow ( Burrow_1.2.2_linux_amd64.tar.gz ) and get fatal error: runtime: out of memory error. Burrow Burrow 1.0.0 run without problems

fatal error: runtime: out of memory

runtime stack:
runtime.throw(0xac5999, 0x16)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/panic.go:608 +0x72
runtime.sysMap(0xc004000000, 0x1650000000, 0x1199b78)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/mem_linux.go:156 +0xc7
runtime.(*mheap).sysAlloc(0x1180200, 0x1650000000, 0x7f08b058f388, 0x7f089a7fbce0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/malloc.go:619 +0x1c7
runtime.(*mheap).grow(0x1180200, 0xb28000, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/mheap.go:920 +0x42
runtime.(*mheap).allocSpanLocked(0x1180200, 0xb28000, 0x1199b88, 0x20300000000000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/mheap.go:848 +0x337
runtime.(*mheap).alloc_m(0x1180200, 0xb28000, 0xffffffffffff0100, 0x7f089a7fbd98)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/mheap.go:692 +0x119
runtime.(*mheap).alloc.func1()
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/mheap.go:759 +0x4c
runtime.(*mheap).alloc(0x1180200, 0xb28000, 0x7f089a010100, 0x4211c5)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/mheap.go:758 +0x8a
runtime.largeAlloc(0x1650000000, 0x450001, 0x7f08b737a6c0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/malloc.go:1019 +0x97
runtime.mallocgc.func1()
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/malloc.go:914 +0x46
runtime.systemstack(0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/asm_amd64.s:351 +0x66
runtime.mstart()
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/proc.go:1229

goroutine 1291 [running]:
runtime.systemstack_switch()
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/asm_amd64.s:311 fp=0xc001806cd8 sp=0xc001806cd0 pc=0x4621c0
runtime.mallocgc(0x1650000000, 0xa5af40, 0x1, 0xc000e2e110)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/malloc.go:913 +0x896 fp=0xc001806d78 sp=0xc001806cd8 pc=0x417096
runtime.newarray(0xa5af40, 0x11000000, 0xa8f720)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/malloc.go:1048 +0x6a fp=0xc001806da8 sp=0xc001806d78 pc=0x41746a
runtime.makeBucketArray(0x9fd920, 0xc000e9901c, 0x0, 0x4, 0x9d1140)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/map.go:355 +0x184 fp=0xc001806de0 sp=0xc001806da8 pc=0x418334
runtime.makemap(0x9fd920, 0x6572726f, 0xc000e990b0, 0x1197b78)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/map.go:321 +0xec fp=0xc001806e28 sp=0xc001806de0 pc=0x41809c
github.com/linkedin/Burrow/core/internal/consumer.decodeMemberAssignmentV0(0xc00011f2d0, 0xc00011f2d0, 0xb6bfe0, 0x1197b78)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:569 +0x12b fp=0xc001806ee8 sp=0xc001806e28 pc=0x8b4a0b
github.com/linkedin/Burrow/core/internal/consumer.decodeMetadataMember(0xc00011f260, 0xc000110001, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:548 +0x428 fp=0xc001806f78 sp=0xc001806ee8 pc=0x8b43f8
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).decodeAndSendGroupMetadata(0xc000272360, 0xc001f50001, 0xc000e2e010, 0xf, 0xc00011f260, 0xc000982660)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:447 +0x9b4 fp=0xc0018074f8 sp=0xc001806f78 pc=0x8b36a4
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).decodeGroupMetadata(0xc000272360, 0xc00011f1f0, 0xc0013cedc2, 0x1b5, 0x22e, 0xc000982600)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:396 +0x658 fp=0xc001807ac8 sp=0xc0018074f8 pc=0x8b23a8
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).processConsumerOffsetsMessage(0xc000272360, 0xc00153f180)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:259 +0x430 fp=0xc001807da0 sp=0xc001807ac8 pc=0x8afa40
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0013b0cf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:171 +0x595 fp=0xc001807fc8 sp=0xc001807da0 pc=0x8ae7a5
runtime.goexit()
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc001807fd0 sp=0xc001807fc8 pc=0x4642a1
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).sendAndReceive(0xc000626840, 0xb6aa80, 0xc0013bb2e0, 0xb63d40, 0xc001cd6410, 0xc00163eb60, 0x53cc995aad670be8)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:622 +0x133
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).GetAvailableOffsets(0xc000626840, 0xc0013bb2e0, 0x1e, 0xc00011f138, 0xc00163ebe8)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:261 +0x70
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).getOffset(0xc0001262d0, 0xac2f4f, 0x12, 0xc00000001e, 0xfffffffffffffffe, 0x15514cf4, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:645 +0x206
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).GetOffset(0xc0001262d0, 0xac2f4f, 0x12, 0xc00000001e, 0xfffffffffffffffe, 0x15514cf4, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:416 +0x60
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).chooseStartingOffset(0xc0013b0d80, 0xfffffffffffffffe, 0xc0017b45a0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:384 +0xb6
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition(0xc000ace100, 0xac2f4f, 0x12, 0x1e, 0xfffffffffffffffe, 0xb6ac00, 0xc0013b0cf0, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:149 +0x1c5
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer(0xc000272360, 0xb6e260, 0xc00004ee20, 0xc00018ac00, 0xb6e440)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:216 +0x986
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).Start(0xc000272360, 0xc000303860, 0xc00028fb28)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:144 +0x378
github.com/linkedin/Burrow/core/internal/helpers.StartCoordinatorModules(0xc000303860, 0xabed22, 0x8)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/helpers/coordinators.go:32 +0x98
github.com/linkedin/Burrow/core/internal/consumer.(*Coordinator).Start(0xc000172400, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/coordinator.go:105 +0x6b
github.com/linkedin/Burrow/core.Start(0xc00016e000, 0xc00001ed80, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/burrow.go:155 +0x42a
main.main()
    /home/travis/gopath/src/github.com/linkedin/Burrow/main.go:115 +0x4d0

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/sigqueue.go:139 +0x9c
os/signal.loop()
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/os/signal/signal_unix.go:29 +0x41

goroutine 18 [chan receive]:
github.com/linkedin/Burrow/vendor/gopkg.in/natefinch/lumberjack%2ev2.(*Logger).millRun(0xc00016e5a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:379 +0x58
created by github.com/linkedin/Burrow/vendor/gopkg.in/natefinch/lumberjack%2ev2.(*Logger).mill.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/gopkg.in/natefinch/lumberjack.v2/lumberjack.go:390 +0x79

goroutine 24 [semacquire]:
sync.runtime_Semacquire(0xc00017a074)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/sema.go:56 +0x39
sync.(*WaitGroup).Wait(0xc00017a074)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/sync/waitgroup.go:130 +0x64
github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).loop(0xc000626000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:538 +0x493
github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.Connect.func1(0xc000626000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:223 +0x2f
created by github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.Connect
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:222 +0x531

goroutine 60 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000626580)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a01b0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 14 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc0006266e0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0003091a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 26 [select]:
github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).sendLoop(0xc000626000, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:792 +0x158
github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).loop.func1(0xc000698240, 0xc000626000, 0xc00017a074)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:509 +0x58
created by github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).loop
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:504 +0x413

goroutine 27 [IO wait]:
internal/poll.runtime_pollWait(0x7f08b731ee30, 0x72, 0xc0001a1bc0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc000194218, 0x72, 0xffffffffffffff00, 0xb65500, 0x1149158)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc000194218, 0xc0006b2000, 0x4, 0x180000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc000194200, 0xc0006b2000, 0x4, 0x180000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_unix.go:169 +0x179
net.(*netFD).Read(0xc000194200, 0xc0006b2000, 0x4, 0x180000, 0xabc7b1ab6a63c, 0x2540be21c, 0xc0001a1cf8)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc00000e008, 0xc0006b2000, 0x4, 0x180000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/net.go:177 +0x68
io.ReadAtLeast(0xb64180, 0xc00000e008, 0xc0006b2000, 0x4, 0x180000, 0x4, 0xa98f80, 0xc0001a1d00, 0xb64180)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/io/io.go:310 +0x88
io.ReadFull(0xb64180, 0xc00000e008, 0xc0006b2000, 0x4, 0x180000, 0x0, 0x1179e00, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/io/io.go:329 +0x58
github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).recvLoop(0xc000626000, 0xb6c400, 0xc00000e008, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:827 +0x15e
github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).loop.func2(0xc000626000, 0xc00017a074)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:523 +0x1f2
created by github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk.(*Conn).loop
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/samuel/go-zookeeper/zk/conn.go:518 +0x459

goroutine 28 [chan receive]:
github.com/linkedin/Burrow/core/internal/zookeeper.(*Coordinator).mainLoop(0xc0001900c0, 0xc000410780)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/zookeeper/coordinator.go:137 +0xba
created by github.com/linkedin/Burrow/core/internal/zookeeper.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/zookeeper/coordinator.go:100 +0x449

goroutine 29 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x0, 0xc00036e000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 30 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x1, 0xc00036e060)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 31 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x2, 0xc00036e0c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 32 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x3, 0xc00036e120)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 33 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x4, 0xc00036e180)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 66 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x5, 0xc00036e1e0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 67 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x6, 0xc00036e240)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 68 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x7, 0xc00036e2a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 69 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x8, 0xc00036e300)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 70 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x9, 0xc00036e360)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 71 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0xa, 0xc00036e3c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 72 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0xb, 0xc00036e420)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 73 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0xc, 0xc00036e480)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 74 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0xd, 0xc00036e4e0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 75 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0xe, 0xc00036e540)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 76 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0xf, 0xc00036e5a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 77 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x10, 0xc00036e600)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 78 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x11, 0xc00036e660)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 79 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x12, 0xc00036e6c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 80 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).requestWorker(0xc0001885a0, 0x13, 0xc00036e780)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:200 +0x7d1
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:156 +0x282

goroutine 81 [chan receive]:
github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).mainLoop(0xc0001885a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:220 +0x96
created by github.com/linkedin/Burrow/core/internal/storage.(*InMemoryStorage).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/inmemory.go:160 +0x37e

goroutine 82 [select]:
github.com/linkedin/Burrow/core/internal/storage.(*Coordinator).mainLoop(0xc0001745a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/coordinator.go:155 +0x1f0
created by github.com/linkedin/Burrow/core/internal/storage.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/storage/coordinator.go:126 +0x14a

goroutine 83 [chan receive]:
github.com/linkedin/Burrow/core/internal/evaluator.(*CachingEvaluator).mainLoop(0xc00018e2d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/evaluator/caching.go:108 +0x82
created by github.com/linkedin/Burrow/core/internal/evaluator.(*CachingEvaluator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/evaluator/caching.go:92 +0x96

goroutine 84 [select]:
github.com/linkedin/Burrow/core/internal/evaluator.(*Coordinator).Start.func1(0xc000172320)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/evaluator/coordinator.go:133 +0x19e
created by github.com/linkedin/Burrow/core/internal/evaluator.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/evaluator/coordinator.go:125 +0x14a

goroutine 39 [IO wait]:
internal/poll.runtime_pollWait(0x7f08b731ef00, 0x72, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc000194018, 0x72, 0xc000688b00, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc000194018, 0xffffffffffffff00, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Accept(0xc000194000, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_unix.go:384 +0x1a0
net.(*netFD).accept(0xc000194000, 0x203000, 0x203000, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc0002a6000, 0xc00002cf70, 0xc00002cf00, 0x416aef)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/tcpsock_posix.go:139 +0x2e
net.(*TCPListener).AcceptTCP(0xc0002a6000, 0xc000419e90, 0x77189279, 0x9393f63412ad1271)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/tcpsock.go:247 +0x47
github.com/linkedin/Burrow/core/internal/httpserver.tcpKeepAliveListener.Accept(0xc0002a6000, 0x45d964b800, 0xc000419eb8, 0x492976, 0x5d3859c9, 0x44037e)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/httpserver/coordinator.go:222 +0x2f
net/http.(*Server).Serve(0xc0001960d0, 0xb69640, 0xc00004e000, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:2826 +0x22f
created by github.com/linkedin/Burrow/core/internal/httpserver.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/httpserver/coordinator.go:187 +0xbd1

goroutine 40 [select]:
github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).responseLoop(0xc000194380)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:396 +0xee
created by github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:264 +0x9b

goroutine 41 [select]:
github.com/linkedin/Burrow/core/internal/helpers.(*PausableTicker).Start.func1(0xc0001727e0, 0xc00016e060, 0xc0003fc000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/helpers/time.go:67 +0xfc
created by github.com/linkedin/Burrow/core/internal/helpers.(*PausableTicker).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/helpers/time.go:65 +0xb3

goroutine 42 [sync.Cond.Wait]:
sync.runtime_notifyListWait(0xc000688050, 0xc000000000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/sema.go:510 +0xeb
sync.(*Cond).Wait(0xc000688040)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/sync/cond.go:56 +0x92
github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).manageEvalLoop(0xc000194380)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:320 +0x5b7
created by github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:275 +0x1a3

goroutine 43 [select]:
github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).tickerLoop(0xc000194380)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:337 +0x101
created by github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:279 +0x1dc

goroutine 56 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc000126000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:679 +0x12b
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).backgroundMetadataUpdater-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:168 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000185070)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.NewClient
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:168 +0x51d

goroutine 54 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/rcrowley/go-metrics.(*meterArbiter).tick(0x1179aa0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/rcrowley/go-metrics/meter.go:239 +0x31
created by github.com/linkedin/Burrow/vendor/github.com/rcrowley/go-metrics.NewMeter
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/rcrowley/go-metrics/meter.go:46 +0x10d

goroutine 59 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000626420)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a00d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 55 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000274f20)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000185010)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 169 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000c14240)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f200)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 133 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000272750)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a0090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 125 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000275b80)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0001855f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 172 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).sendAndReceive(0xc000626840, 0xb6a800, 0xc0013bb200, 0xb63c00, 0xc0013bb260, 0xc000a04db0, 0x11419f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:622 +0x133
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Fetch(0xc000626840, 0xc0013bb200, 0x18, 0xc00047ad80, 0xc00163ea48)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:291 +0x70
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).fetchNewMessages(0xc000e705a0, 0x0, 0x0, 0x0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:806 +0x3f7
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).subscriptionConsumer(0xc000e705a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:692 +0x124
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).subscriptionConsumer-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:633 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f270)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).newBrokerConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:633 +0x19c

goroutine 150 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000272750)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 113 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc0002758c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000184100)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 134 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000272750)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a00a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 15 [select]:
github.com/linkedin/Burrow/core/internal/cluster.(*KafkaCluster).mainLoop(0xc000194900, 0xb6e260, 0xc00004ef10)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/cluster/kafka_cluster.go:123 +0x11d
created by github.com/linkedin/Burrow/core/internal/cluster.(*KafkaCluster).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/cluster/kafka_cluster.go:101 +0x413

goroutine 175 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0014ce000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f810)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 162 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc0001262d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:679 +0x12b
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).backgroundMetadataUpdater-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:168 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004ed40)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.NewClient
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:168 +0x51d

goroutine 92 [select]:
github.com/linkedin/Burrow/core/internal/cluster.(*KafkaCluster).mainLoop(0xc000194680, 0xb6e260, 0xc000185090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/cluster/kafka_cluster.go:123 +0x11d
created by github.com/linkedin/Burrow/core/internal/cluster.(*KafkaCluster).Start
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/cluster/kafka_cluster.go:101 +0x413

goroutine 61 [runnable]:
time.Sleep(0xf4240)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/time.go:105 +0x14f
github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).sendEvaluatorRequests(0xc000194380)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:388 +0x401
created by github.com/linkedin/Burrow/core/internal/notifier.(*Coordinator).manageEvalLoop
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/notifier/coordinator.go:315 +0x357

goroutine 104 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000275340)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a0fb0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 112 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000275760)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000184100)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 114 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc000272240)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:679 +0x12b
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*client).backgroundMetadataUpdater-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:168 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004eef0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.NewClient
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/client.go:168 +0x51d

goroutine 176 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0014ce000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f820)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 171 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).subscriptionManager(0xc000e705a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:659 +0x399
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).subscriptionManager-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:632 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f220)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).newBrokerConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:632 +0x14e

goroutine 127 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000c14240)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 132 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000275a20)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004eea0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 170 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000c14240)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f210)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 168 [runnable]:
internal/poll.runtime_pollWait(0x7f08b731ebc0, 0x72, 0xc000bb8930)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc000fe0718, 0x72, 0xffffffffffffff00, 0xb65500, 0x1149158)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc000fe0718, 0xc000b0c000, 0x4000, 0x4000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc000fe0700, 0xc000b0c000, 0x4000, 0x4000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_unix.go:169 +0x179
net.(*netFD).Read(0xc000fe0700, 0xc000b0c000, 0x4000, 0x4000, 0x4138ab, 0xc000010000, 0xa19d80)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0001700d8, 0xc000b0c000, 0x4000, 0x4000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/net.go:177 +0x68
crypto/tls.(*block).readFromUntil(0xc000a04ab0, 0xb64180, 0xc0001700d8, 0x5, 0xc0001700d8, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/crypto/tls/conn.go:492 +0x89
crypto/tls.(*Conn).readRecord(0xc000667500, 0xae1517, 0xc000667620, 0x6fc23a988)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/crypto/tls/conn.go:593 +0xdd
crypto/tls.(*Conn).Read(0xc000667500, 0xc000eb4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/crypto/tls/conn.go:1145 +0xf1
bufio.(*Reader).Read(0xc000a00660, 0xc000d1c880, 0x8, 0x8, 0xc000bb8d48, 0x5351c8, 0xc000fe0700)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/bufio/bufio.go:216 +0x22f
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*bufConn).Read(0xc00143ef00, 0xc000d1c880, 0x8, 0x8, 0x0, 0xc000bb8d88, 0x4138ab)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:109 +0x4d
io.ReadAtLeast(0x7f08b0533488, 0xc00143ef00, 0xc000d1c880, 0x8, 0x8, 0x8, 0xa69b60, 0x0, 0x7f08b0533488)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/io/io.go:310 +0x88
io.ReadFull(0x7f08b0533488, 0xc00143ef00, 0xc000d1c880, 0x8, 0x8, 0x0, 0x1179e00, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/io/io.go:329 +0x58
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc000626840)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:707 +0x29d
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00004f170)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 151 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000272870)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e720)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 149 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver(0xc0006262c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:694 +0xdc
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).responseReceiver-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e690)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*Broker).Open.func1
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/broker.go:150 +0x809

goroutine 135 [select]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).subscriptionManager(0xc000841db0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:659 +0x399
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*brokerConsumer).subscriptionManager-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:632 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a00c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).newBrokerConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:632 +0x14e

goroutine 136 [running]:
    goroutine running on other thread; stack unavailable
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).newBrokerConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:633 +0x19c

goroutine 180 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0001265a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000308af0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 181 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0001265a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000308b00)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 182 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0001265a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 152 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000272870)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e730)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 153 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000272870)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 137 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000126630)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a0430)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 138 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000126630)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0002a0450)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 139 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000126630)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 177 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0014ce000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 183 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000c143f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000308be0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 184 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000c143f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000308bf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 185 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000c143f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 154 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000272900)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e830)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 155 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000272900)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e840)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 156 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000272900)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 157 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0001266c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e880)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 158 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0001266c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00034e890)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 159 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0001266c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 186 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000126750)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000308100)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 187 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000126750)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000308110)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 188 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000126750)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1351 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0015de090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045def0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1329 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0015de000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b08c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1346 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc00123ebd0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1278 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0013b0a20)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd6130)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1347 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000c15ef0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045ddf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1289 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0013b0cf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0012dc920)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1362 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0015de000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1356 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000273680)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0017b8000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1354 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0002735f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045df60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1348 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000c15ef0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045de00)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1357 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000273680)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0017b8010)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1265 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc00123ebd0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045dce0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1350 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0015de090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045dee0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1353 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0002735f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045df50)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1352 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0015de090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1280 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0013b0a20)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1323 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc00123ecf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b07d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1349 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000c15ef0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1358 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000273680)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1355 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0002735f0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1332 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0002734d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd6220)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1262 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc00123eb40)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045dc40)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1263 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc00123eb40)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1283 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000273440)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0012dc540)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 937 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000127170)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000149b70)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1330 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0013b0c60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd61b0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1287 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0015de120)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0012dc8e0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1331 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0013b0c60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 818 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0014ce090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1261 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc00123eb40)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045dc30)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1264 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc00123ebd0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00045dcd0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1288 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0015de120)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 672 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0014ce090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00039bd00)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1285 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000273440)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1290 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0013b0cf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0012dc930)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1286 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0015de120)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0012dc8d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1202 [IO wait]:
internal/poll.runtime_pollWait(0x7f08b731e200, 0x72, 0xc0017f9b90)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc00102ff18, 0x72, 0xffffffffffffff00, 0xb65500, 0x1149158)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc00102ff18, 0xc000860000, 0x1000, 0x1000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc00102ff00, 0xc000860000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_unix.go:169 +0x179
net.(*netFD).Read(0xc00102ff00, 0xc000860000, 0x1000, 0x1000, 0x5222fb, 0xc0017f9ca0, 0x6b6b8a)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0002a7068, 0xc000860000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/net.go:177 +0x68
net/http.(*connReader).Read(0xc00221ffb0, 0xc000860000, 0x1000, 0x1000, 0xbf46349d5eb04273, 0x46b1ddb3a7, 0x1179e00)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:786 +0xfc
bufio.(*Reader).fill(0xc002354a80)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc002354a80, 0x4, 0x46b1ddb3a7, 0x1179e00, 0x0, 0x0, 0x1179e00)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/bufio/bufio.go:132 +0x3f
net/http.(*conn).serve(0xc000b09e00, 0xb69b00, 0xc000ebb940)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:1872 +0x7ba
created by net/http.(*Server).Serve
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:2851 +0x2f5

goroutine 673 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0014ce090)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc00039bd10)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1279 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0013b0a20)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd6140)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 939 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000127170)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1327 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000273560)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 938 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000127170)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc000149b80)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1336 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000c15e60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd62d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1326 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000273560)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b0850)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1334 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc0002734d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1284 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000273440)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0012dc550)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1322 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc00123ecf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b07c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1364 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc000273710)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b09d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1337 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000c15e60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1321 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc00123ec60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1281 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0013b0c60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd61a0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1328 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc0015de000)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b08b0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1365 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc000273710)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1333 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc0002734d0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd6230)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300

goroutine 1324 [select]:
github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).partitionConsumer(0xc000272360, 0xb6ac00, 0xc00123ecf0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:169 +0x1b6
created by github.com/linkedin/Burrow/core/internal/consumer.(*KafkaClient).startKafkaConsumer
    /home/travis/gopath/src/github.com/linkedin/Burrow/core/internal/consumer/kafka_client.go:226 +0x91a

goroutine 1325 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000273560)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b0840)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1229 [IO wait]:
internal/poll.runtime_pollWait(0x7f08b0556df0, 0x72, 0xc001ced858)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/runtime/netpoll.go:173 +0x66
internal/poll.(*pollDesc).wait(0xc001b84e18, 0x72, 0xffffffffffffff00, 0xb65500, 0x1149158)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0x9a
internal/poll.(*pollDesc).waitRead(0xc001b84e18, 0xc000e62000, 0x1000, 0x1000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc001b84e00, 0xc000e62000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/internal/poll/fd_unix.go:169 +0x179
net.(*netFD).Read(0xc001b84e00, 0xc000e62000, 0x1000, 0x1000, 0xc0010a0400, 0xc001ced968, 0x6b6bb0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0003f8ca8, 0xc000e62000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/net.go:177 +0x68
net/http.(*connReader).Read(0xc000e26030, 0xc000e62000, 0x1000, 0x1000, 0xc0000ac300, 0xace1d1, 0x23)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:786 +0xfc
bufio.(*Reader).fill(0xc00238f4a0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).ReadSlice(0xc00238f4a0, 0xc001730b0a, 0x455a28, 0xc001cedab8, 0x454d5f, 0x117e380, 0x203000)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/bufio/bufio.go:341 +0x36
bufio.(*Reader).ReadLine(0xc00238f4a0, 0xc001730b00, 0x100, 0xf8, 0xaabfe0, 0xc0002b4000, 0xc001cedb18)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/bufio/bufio.go:370 +0x34
net/textproto.(*Reader).readLineSlice(0xc00172fbf0, 0xc00220a580, 0xc001cedb98, 0x417378, 0x100, 0xaabfe0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/textproto/reader.go:55 +0x6f
net/textproto.(*Reader).ReadLine(0xc00172fbf0, 0xc001730b00, 0xc001095200, 0xc000191f00, 0x1179e00)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/textproto/reader.go:36 +0x2b
net/http.readRequest(0xc00238f4a0, 0xae0d00, 0xc001730b00, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/request.go:958 +0x8c
net/http.(*conn).readRequest(0xc000fb1a40, 0xb69b00, 0xc0010a03c0, 0x0, 0x0, 0x0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:966 +0x162
net/http.(*conn).serve(0xc000fb1a40, 0xb69b00, 0xc0010a03c0)
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:1788 +0x49e
created by net/http.(*Server).Serve
    /home/travis/.gimme/versions/go1.11.5.linux.amd64/src/net/http/server.go:2851 +0x2f5

goroutine 1319 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc00123ec60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b0640)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1335 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000c15e60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc001cd62c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1363 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher(0xc000273710)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:336 +0x5a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).dispatcher-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b09c0)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:163 +0x2ae

goroutine 1320 [chan receive]:
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder(0xc00123ec60)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:451 +0x27a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*partitionConsumer).responseFeeder-fm()
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x2a
github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.withRecover(0xc0021b0650)
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/utils.go:45 +0x43
created by github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama.(*consumer).ConsumePartition
    /home/travis/gopath/src/github.com/linkedin/Burrow/vendor/github.com/Shopify/sarama/consumer.go:164 +0x300
farvour commented 5 years ago

We also seem to be experiencing some out of memory issues. I'm going to try and pull a stack trace and see if it's similar to this one.

jonbuffington commented 5 years ago

In our usage, burrow's memory peak usage at start-up is substantially larger (e.g., >230M) than stable running (32M). Later builds against master reduce the start-up peak usage (80M) but do not eliminate the issue.

stefan-as commented 4 years ago

Same issue, same stack trace.

stefan@messaging-staging-worker-00:~$ free
              total        used        free      shared  buff/cache   available
Mem:        8170320     2478996     4189312        3848     1502012     5419388