Cannot Start Redpanda After Upgrading from 22.2.7 to 22.3.2 (or 22.3.1)

redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

9.39k stars 577 forks source link

total used free shared buff/cache available Mem: 251G 99G 144G 4.1G 7.9G 146G Swap: 4.0G 0B 4.0G

Nov 16 15:35:47 [srv-01-03-404.iad1.trmr.io](http://srv-01-03-404.iad1.trmr.io/) kernel: traps: redpanda[43550] trap invalid opcode ip:560abdefcfc5 sp:7fff61ea93d0 error:0 in redpanda[560abb453000+39a9000]

Nov 16 16:02:52 srv-01-03-404.iad1.trmr.io rpk[807]: ERROR 2022-11-16 16:02:52,255 [shard 0] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-xfs-builders-i-06c4550a08f45c0ae-1/redpanda/redpanda/src/v/cluster/scheduling/allocation_node.cc:81) 'domain_partitions > allocation_capacity{0} && domain_partitions <= _allocated_partitions' Unable to deallocate partition from core 27 in domain 0 at node {node: 8, max_partitions_per_core: 7000, state: active, partition_capacity: 447993, weights: [(2)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(1)(1)(1)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)], allocated: 5({0: 0, -1: 5})} Nov 16 16:02:52 srv-01-03-404.iad1.trmr.io rpk[807]: ERROR 2022-11-16 16:02:52,255 [shard 0] assert - Backtrace below: Nov 16 16:02:52 srv-01-03-404.iad1.trmr.io rpk[807]: 0x534d444 0x4504f94 0x402093d 0x412ddc4 0x50ea0ef 0x50eddc7 0x50eb199 0x500bd31 0x5009e4f 0x1a6073e 0x53ffbc9 /opt/redpanda/lib/libc.so.6+0x2d58f /opt/redpanda/lib/libc.so.6+0x2d648 0x1a5b524

Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: Reactor stalled for 111 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450be8a 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1492a1b 0xffffffffa14932ef 0xffffffffa149363a 0xffffffffa13cf672 0xffffffffa13cfac2 0xffffffffa13d022e 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1ac874c 0xffffffffa10fc410 0xffffffffa10fc483 0xffffffffc01b7151 0xffffffffa17db0b0 0xffffffffa17db24d 0xffffffffa14929fb 0xffffffffa1492f15 0xffffffffa14932ef 0xffffffffa149363a 0xffffffffa13cf672 0xffffffffa13cfac2 0xffffffffa13d022e 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffc01b7055 0xffffffffa17db24d 0xffffffffa14929fb 0xffffffffa1492f15 0xffffffffa14932ef 0xffffffffa149363a 0xffffffffa13cf672 0xffffffffa13cfac2 0xffffffffa13d022e 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa18434c7 0xffffffffa1843316 0xffffffffa184337c 0xffffffffa184357f 0xffffffffa1843316 0xffffffffa1847c9c 0xffffffffa1863038 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1895b9e 0xffffffffa1895ef2 0xffffffffc016fa50 0xffffffffc0170358 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffc02d2d22 0xffffffffc02d3d98 0xffffffffa13caa81 0xffffffffa13cff06 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffc016feed 0xffffffffc0170358 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffc01b62d1 0xffffffffc01b6bcc 0xffffffffc01b7138 0xffffffffa17db0b0 0xffffffffa17db24d 0xffffffffa14929fb 0xffffffffa1492f15 0xffffffffa14932ef 0xffffffffa149363a 0xffffffffa13cf672 0xffffffffa13cfac2 0xffffffffa13d022e 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1895e14 0xffffffffa1895ef2 0xffffffffc016fa50 0xffffffffc0170358 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa14a1f52 0xffffffffa14a2553 0xffffffffa1492a2c 0xffffffffa149308a 0xffffffffa14932ef 0xffffffffc01b2864 0xffffffffa1493af7 0xffffffffa1493d9a 0xffffffffa13b5beb 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1842403 0xffffffffc016feb2 0xffffffffc0170358 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab48be 0xffffffffa1c00d67 0xffffffffa12697ca 0xffffffffc009fbb9 0xffffffffc00a050e 0xffffffffa14a0b45 0xffffffffa14a218a 0xffffffffa14a7923 0xffffffffa14a1d82 0xffffffffa1493b20 0xffffffffa1493d9a 0xffffffffa13b5beb 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1ac881d 0xffffffffa10fc43a 0xffffffffa10fca30 0xffffffffa1839e90 0xffffffffa192ef2f 0xffffffffa1931004 0xffffffffa193f779 0xffffffffa1941ae1 0xffffffffa190d827 0xffffffffa190daa6 0xffffffffa190db3f 0xffffffffa190dd39 0xffffffffa190dedd 0xffffffffa190e19a 0xffffffffa186218a 0xffffffffa186238a 0xffffffffa1862afa 0xffffffffc01703cc 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1a432a4 0xffffffffa1860e06 0xffffffffa186201c 0xffffffffa186238a 0xffffffffa1895895 0xffffffffa1895991 0xffffffffa1862baa 0xffffffffc01703cc 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa13099a4 0xffffffffc01b71f6 0xffffffffa12697c4 0xffffffffc01b70e6 0xffffffffa17db0b0 0xffffffffa17db24d 0xffffffffa14929fb 0xffffffffa1492f15 0xffffffffa14932ef 0xffffffffa149363a 0xffffffffa13cf672 0xffffffffa13cfac2 0xffffffffa13d022e 0xffffffffa13d0511 0xffffffffc02c6f6a 0xffffffffc02c7135 0xffffffffa13b177f 0xffffffffa13b2027 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa18cdd04 0xffffffffc016f465 0xffffffffc016fa35 0xffffffffc0170358 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa14b649b 0xffffffffc01b2864 0xffffffffa1493af7 0xffffffffa1493d9a 0xffffffffa13b5beb 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa19353ed 0xffffffffa19368b6 0xffffffffa193988c 0xffffffffa1928672 0xffffffffa1930ff4 0xffffffffa193f779 0xffffffffa1941ae1 0xffffffffa190d827 0xffffffffa190daa6 0xffffffffa190db3f 0xffffffffa190dd39 0xffffffffa190dedd 0xffffffffa190e19a 0xffffffffa186218a 0xffffffffa186238a 0xffffffffa1895895 0xffffffffa1895991 0xffffffffa1862baa 0xffffffffc01703cc 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa1ac881d 0xffffffffa10fc43a 0xffffffffa10fc483 0xffffffffc01b7151 0xffffffffa17db0b0 0xffffffffa17db24d 0xffffffffa14929fb 0xffffffffa1492f15 0xffffffffa14932ef 0xffffffffa149363a 0xffffffffa13cf672 0xffffffffa13cfac2 0xffffffffa13d022e 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffc016f82f 0xffffffffc0170358 0xffffffffa1862c81 0xffffffffa18631c2 0xffffffffa1e000df 0xffffffffa109c8bf 0xffffffffa1ab487a 0xffffffffa1c00d67 Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: Reactor stalled for 168 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbed5 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: Reactor stalled for 276 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450d36d 0x450c2e2 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO 2022-11-16 16:14:50,622 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: WARN 2022-11-16 16:14:50,743 [shard 27] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: -------- Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void>

Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: WARN 2022-11-16 16:14:50,743 [shard 27] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: -------- Nov 16 16:14:50 srv-01-03-404.iad1.trmr.io rpk[5055]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void>

[Thu Nov 10 14:09:03 2022] reactor-55[39825]: segfault at 196964 ip 000055dac0dded8e sp 00007fb01a3efe00 error 4 in redpanda[55dabe7b2000+353c000] [Thu Nov 10 14:09:03 2022] Code: 00 00 48 8b 81 80 00 00 00 48 8b 89 88 00 00 00 48 39 c8 0f 84 27 01 00 00 49 8b 55 78 49 8b b5 90 00 00 00 48 ff ce 48 21 c6 <4c> 8b 34 f2 41 80 7e 30 00 0f 85 07 01 00 00 49 8b 06 48 ff c0 49 [Sat Nov 12 21:14:04 2022] reactor-56[35909]: segfault at 1846ec0ac34 ip 000055a016e04893 sp 00007f986c1ef720 error 4 in redpanda[55a013c7b000+353c000] [Sat Nov 12 21:14:04 2022] Code: 0f 44 c9 c7 01 00 00 00 00 41 89 30 48 8b 4c 07 18 48 85 c9 74 b1 4c 8d 54 07 18 0f b7 7c 07 02 0f 1f 80 00 00 00 00 48 89 cb <48> 8b 09 48 89 2b 48 ff c2 ff c7 48 89 dd 48 85 c9 75 ea 49 c7 02 [Wed Nov 16 15:31:43 2022] traps: redpanda[38884] trap invalid opcode ip:55b81812efc5 sp:7ffd31533cd0 error:0 in redpanda[55b815685000+39a9000] [Wed Nov 16 15:31:46 2022] traps: redpanda[39058] trap invalid opcode ip:55748cca6fc5 sp:7ffe08730b60 error:0 in redpanda[55748a1fd000+39a9000] [Wed Nov 16 15:31:49 2022] traps: redpanda[39257] trap invalid opcode ip:55a18897afc5 sp:7ffcca2be110 error:0 in redpanda[55a185ed1000+39a9000] . . . <lots more the same trap message>

Nov 16 15:17:39 srv-01-11-301.iad1.trmr.io rpk[55700]: WARN 2022-11-16 15:17:39,429 [shard 59] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 16 15:17:39 srv-01-11-301.iad1.trmr.io rpk[55700]: -------- Nov 16 15:17:39 srv-01-11-301.iad1.trmr.io rpk[55700]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void>

-- Logs begin at Thu 2022-11-17 08:45:50 EST. -- Nov 17 08:48:10 srv-01-03-404.iad1.trmr.io systemd[1]: Starting Redpanda, the fastest queue in the West.... Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: System check 'Dir '/var/lib/redpanda/data' IRQs affinity set' failed. Required: true, Current false Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: System check 'NTP Synced' failed. Required: true, Current false Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: System check - PASSED Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: We'd love to hear about your experience with redpanda: Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: https://redpanda.com/feedback Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Starting redpanda... Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Running: Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: SHELL=/sbin/nologin USER=redpanda PATH=/opt/redpanda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin PWD=/ LANG=en_US.UTF-8 NOTIFY_SOCKET=/run/systemd/notify SHLVL=0 HOME=/var/lib/redpanda LOGNAME=redpanda START_ARGS=--check=true /opt/redpanda/bin/redpanda redpanda --redpanda-cfg /etc/redpanda/redpanda.yaml --lock-memory=false Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Welcome to the Redpanda community! Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Documentation: https://docs.redpanda.com - Product documentation site Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: GitHub Discussion: https://github.com/redpanda-data/redpanda/discussions - Longer, more involved discussions Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: GitHub Issues: https://github.com/redpanda-data/redpanda/issues - Report and track issues with the codebase Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Support: https://support.redpanda.com - Contact the support team privately Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Product Feedback: https://redpanda.com/feedback - Let us know how we can improve your experience Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Slack: https://redpanda.com/slack - Chat about all things Redpanda. Join the conversation! Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: Twitter: https://twitter.com/redpandadata - All the latest Redpanda news! Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,740 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,740 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,740 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,741 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,829 [shard 0] main - application.cc:299 - Redpanda v22.3.3 - 41a324dfcf816dbf656de83744146e89ac3c2583-dirty Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,830 [shard 0] main - application.cc:307 - kernel=6.0.9-1.el7.elrepo.x86_64, nodename=srv-01-03-404.iad1.trmr.io, machine=x86_64 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,830 [shard 0] main - application.cc:235 - System resources: { cpus: 64, available memory: 233.625GiB, reserved memory: 16.354GiB} Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,830 [shard 0] main - application.cc:243 - File handle limit: 800000/800000 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,852 [shard 0] main - application.cc:539 - Cluster configuration properties: Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,852 [shard 0] main - application.cc:540 - (use `rpk cluster config edit` to change) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.abort_index_segment_size:50000 - Capacity (in number of txns) of an abort index segment Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.abort_timed_out_transactions_interval_ms:10000 - How often look for the inactive transactions and abort them Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.admin_api_require_auth:0 - Whether admin API clients must provide HTTP Basic authentication headers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.aggregate_metrics:0 - Enable aggregations of metrics returned by the prometheus '/metrics' endpoint. Metric aggregation is performed by summing the values of samples by labels. Aggregations are performed where it makes sense by the shard and/or partition labels. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.alter_topic_cfg_timeout_ms:5000 - Time to wait for entries replication in controller log when executing alter configuration requst Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.append_chunk_size:16384 - Size of direct write operations to disk in bytes Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.auto_create_topics_enabled:0 - Allow topic auto creation Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_access_key:{nullopt} - AWS access key Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_api_endpoint:{nullopt} - Optional API endpoint Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_api_endpoint_port:443 - TLS port override Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_bucket:{nullopt} - AWS bucket that should be used to store data Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_cache_check_interval:30000 - Timeout to check if cache eviction should be triggered Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_cache_size:21474836480 - Max size of archival cache Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_credentials_source:config_file - The source of credentials to connect to cloud services Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_disable_tls:0 - Disable TLS for all S3 connections Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_enable_compacted_topic_reupload:1 - Enable re-uploading data for compacted topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_enable_remote_read:0 - Default remote read config value for new topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_enable_remote_write:0 - Default remote write value for new topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_enabled:0 - Enable archival storage Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_housekeeping_interval_ms:300000 - Interval for cloud storage housekeeping tasks Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_initial_backoff_ms:100 - Initial backoff time for exponetial backoff algorithm (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_manifest_upload_timeout_ms:10000 - Manifest upload timeout (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_max_connection_idle_time_ms:5000 - Max https connection idle time (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_max_connections:20 - Max number of simultaneous connections to S3 per shard (includes connections used for both uploads and downloads) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_max_materialized_segments_per_shard:{nullopt} - Maximum concurrent readers of remote data per CPU core. If unset, value of `topic_partitions_per_shard` multiplied by 2 is used. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_max_readers_per_shard:{nullopt} - Maximum concurrent readers of remote data per CPU core. If unset, value of `topic_partitions_per_shard` is used, i.e. one reader per partition if the shard is at its maximum partition capacity. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_max_segments_pending_deletion_per_partition:5000 - The per-partition limit for the number of segments pending deletion from the cloud. Segments can be deleted due to retention or compaction. If this limit is breached and deletion fails, then segments will be orphaned in the cloud and will have to be removed manually Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_metadata_sync_timeout_ms:10000 - Timeout for SI metadata synchronization Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_readreplica_manifest_sync_timeout_ms:30000 - Timeout to check if new data is available for partition in S3 for read replica Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_reconciliation_interval_ms:1000 - Interval at which the archival service runs reconciliation (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_region:{nullopt} - AWS region that houses the bucket used for storage Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_secret_key:{nullopt} - AWS secret key Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_segment_max_upload_interval_sec:{nullopt} - Time that segment can be kept locally without uploading it to the remote storage (sec) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_segment_upload_timeout_ms:30000 - Log segment upload timeout (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_trust_file:{nullopt} - Path to certificate that should be used to validate server certificate during TLS handshake Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_ctrl_d_coeff:0 - derivative coefficient for upload PID controller. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_ctrl_max_shares:1000 - maximum number of IO and CPU shares that archival upload can use Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_ctrl_min_shares:100 - minimum number of IO and CPU shares that archival upload can use Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_ctrl_p_coeff:-2 - proportional coefficient for upload PID controller Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_ctrl_update_interval_ms:60000 - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_loop_initial_backoff_ms:100 - Initial backoff interval when there is nothing to upload for a partition (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cloud_storage_upload_loop_max_backoff_ms:10000 - Max backoff interval when there is nothing to upload for a partition (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.cluster_id:{11292c39-9267-4cd3-9d74-d7f63648aee0} - Cluster identifier Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compacted_log_segment_size:268435456 - How large in bytes should each compacted log segment be (default 256MiB) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_backlog_size:{nullopt} - target backlog size for compaction controller. if not set compaction target compaction backlog would be equal to Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_d_coeff:0.2 - derivative coefficient for compaction PID controller. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_i_coeff:0 - integral coefficient for compaction PID controller. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_max_shares:1000 - maximum number of IO and CPU shares that compaction process can use Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_min_shares:10 - minimum number of IO and CPU shares that compaction process can use Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_p_coeff:-12.5 - proportional coefficient for compaction PID controller. This has to be negative since compaction backlog should decrease when number of compaction shares increases Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.compaction_ctrl_update_interval_ms:30000 - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.controller_backend_housekeeping_interval_ms:1000 - Interval between iterations of controller backend housekeeping loop Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.controller_log_accummulation_rps_capacity_acls_and_users_operations:{nullopt} - Maximum capacity of rate limit accumulationin controller acls and users operations limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.controller_log_accummulation_rps_capacity_configuration_operations:{nullopt} - Maximum capacity of rate limit accumulationin controller configuration operations limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.controller_log_accummulation_rps_capacity_move_operations:{nullopt} - Maximum capacity of rate limit accumulationin controller move operations limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.controller_log_accummulation_rps_capacity_node_management_operations:{nullopt} - Maximum capacity of rate limit accumulationin controller node management operations limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.controller_log_accummulation_rps_capacity_topic_operations:{nullopt} - Maximum capacity of rate limit accumulationin controller topic operations limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.coproc_max_batch_size:32768 - Maximum amount of bytes to read from one topic read Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.coproc_max_inflight_bytes:10485760 - Maximum amountt of inflight bytes when sending data to wasm engine Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.coproc_max_ingest_bytes:655360 - Maximum amount of data to hold from input logs in memory Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.coproc_offset_flush_interval_ms:300000 - Interval for which all coprocessor offsets are flushed to disk Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.create_topic_timeout_ms:2000 - Timeout (ms) to wait for new topic creation Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.default_num_windows:10 - Default number of quota tracking windows Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.default_topic_partitions:10 - Default number of partitions per topic Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.default_topic_replications:3 - Default replication factor for new topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.default_window_sec:1000 - Default quota tracking window size in milliseconds Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.delete_retention_ms:86400000 - delete segments older than this - default 1 week Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.disable_batch_cache:0 - Disable batch cache in log manager Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.disable_metrics:0 - Disable registering metrics exposed on the internal metrics endpoint (/metrics) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.disable_public_metrics:0 - Disable registering metrics exposed on the public metrics endpoint (/public_metrics) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.election_timeout_ms:1500 - Election timeout expressed in milliseconds Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.enable_admin_api: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.enable_auto_rebalance_on_node_add:0 - Enable automatic partition rebalancing when new nodes are added Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,853 [shard 0] main - application.cc:515 - redpanda.enable_controller_log_rate_limiting:0 - Enables limiting of controller log write rate Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_coproc:0 - Enable coprocessing mode Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_idempotence:1 - Enable idempotent producer Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_leader_balancer:1 - Enable automatic leadership rebalancing Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_metrics_reporter:1 - Enable cluster metrics reporter Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_pid_file:1 - Enable pid file. You probably don't want to change this. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_rack_awareness:0 - Enables rack-aware replica assignment Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_sasl:0 - Enable SASL authentication for Kafka connections, authorization is required. see also `kafka_enable_authorization` Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.enable_transactions:1 - Enable transactions Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.features_auto_enable:1 - Whether new feature flags may auto-activate after upgrades (true) or must wait for manual activation via the admin API (false) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.fetch_max_bytes:57671680 - Maximum number of bytes returned in fetch request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.fetch_reads_debounce_timeout:1 - Time to wait for next read in fetch request when requested min bytes wasn't reached Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.fetch_session_eviction_timeout_ms:60000 - Minimum time before which unused session will get evicted from sessions. Maximum time after which inactive session will be deleted is two time given configuration valuecache Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.full_raft_configuration_recovery_pattern:{} - Recover raft configuration on start for NTPs matching pattern Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.group_initial_rebalance_delay:3000 - Extra delay (ms) added to rebalance phase to wait for new members Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.group_max_session_timeout_ms:300000 - The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.group_min_session_timeout_ms:6000 - The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection at the cost of more frequent consumer heartbeating, which can overwhelm broker resources. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.group_new_member_join_timeout:30000 - Timeout for new member joins Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.group_topic_partitions:16 - Number of partitions in the internal group membership topic Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.health_manager_tick_interval:180000 - How often the health manager runs Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.health_monitor_max_metadata_age:10000 - Max age of metadata cached in the health monitor of non controller node Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.health_monitor_tick_interval:10000 - How often health monitor refresh cluster state Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.id_allocator_batch_size:1000 - Id allocator allocates messages in batches (each batch is a one log record) and then serves requests from memory without touching the log until the batch is exhausted. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.id_allocator_log_capacity:100 - Capacity of the id_allocator log in number of messages. Once it reached id_allocator_stm should compact the log. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.id_allocator_replication: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.internal_topic_replication_factor:3 - Target replication factor for internal topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.join_retry_timeout_ms:5000 - Time between cluster join retries in milliseconds Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_admin_topic_api_rate:{nullopt} - Target quota rate (partition mutations per default_window_sec) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_batch_max_bytes:1048576 - Maximum size of a batch processed by server. If batch is compressed the limit applies to compressed batch size Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_connection_rate_limit:{nullopt} - Maximum connections per second for one core Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_connection_rate_limit_overrides:{} - Overrides for specific ips for maximum connections per second for one core Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_connections_max:{nullopt} - Maximum number of Kafka client connections per broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_connections_max_overrides:{} - Per-IP overrides of kafka connection count limit, list of <ip>:<count> strings Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_connections_max_per_ip:{nullopt} - Maximum number of Kafka client connections from each IP address, per broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_enable_authorization:{nullopt} - Enable authorization for Kafka connections. Values:- `nil`: Ignored. Authorization is enabled with `enable_sasl: true`; `true`: authorization is required; `false`: authorization is disabled. See also: `enable_sasl` and `kafka_api[].authentication_method` Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_group_recovery_timeout_ms:30000 - Kafka group recovery timeout expressed in milliseconds Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_max_bytes_per_fetch:67108864 - Limit fetch responses to this many bytes, even if total of partition bytes limits is higher Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_mtls_principal_mapping_rules:{{}} - Principal Mapping Rules for mTLS Authentication on the Kafka API Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_depth_alpha:0.8 - Smoothing factor for kafka queue depth control depth tracking. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_depth_update_ms:7000 - Update frequency for kafka queue depth control. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_enable:0 - Enable kafka queue depth control. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_idle_depth:10 - Queue depth when idleness is detected in kafka queue depth control. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_latency_alpha:0.002 - Smoothing parameter for kafka queue depth control latency tracking. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_max_depth:100 - Maximum queue depth used in kafka queue depth control. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_max_latency_ms:80 - Max latency threshold for kafka queue depth control depth tracking. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_min_depth:1 - Minimum queue depth used in kafka queue depth control. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_window_count:12 - Number of windows used in kafka queue depth control latency tracking. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_qdc_window_size_ms:1500 - Window size for kafka queue depth control latency tracking. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_request_max_bytes:104857600 - Maximum size of a single request processed via Kafka API Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_rpc_server_stream_recv_buf:{nullopt} - Userspace receive buffer max size in bytes Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_rpc_server_tcp_recv_buf:{nullopt} - Kafka server TCP receive buffer size in bytes. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kafka_rpc_server_tcp_send_buf:{nullopt} - Kafka server TCP transmit buffer size in bytes. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kvstore_flush_interval:10 - Key-value store flush interval (ms) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.kvstore_max_segment_size:16777216 - Key-value maximum segment size (bytes) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.leader_balancer_idle_timeout:120000 - Leadership rebalancing idle timeout Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.leader_balancer_mute_timeout:300000 - Leadership rebalancing mute timeout Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.leader_balancer_transfer_limit_per_shard:512 - Per shard limit for in progress leadership transfers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_cleanup_policy:delete - Default topic cleanup policy Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_compaction_interval_ms:10000 - How often do we trigger background compaction Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_compression_type:producer - Default topic compression type Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_message_timestamp_type:CreateTime - Default topic messages timestamp type Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_segment_size:104857600 - Default log segment size in bytes for topics which do not set segment.bytes Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_segment_size_jitter_percent:5 - Random variation to the segment size limit used for each partition Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_segment_size_max:{nullopt} - Upper bound on topic segment.bytes: higher values will be clamped to this limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.log_segment_size_min:{nullopt} - Lower bound on topic segment.bytes: lower values will be clamped to this limit Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.max_compacted_log_segment_size:5368709120 - Max compacted segment size after consolidation Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.max_kafka_throttle_delay_ms:60000 - Fail-safe maximum throttle delay on kafka requests Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.max_version: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.members_backend_retry_ms:5000 - Time between members backend reconciliation loop retries Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metadata_dissemination_interval_ms:3000 - Interaval for metadata dissemination batching Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metadata_dissemination_retries:30 - Number of attempts of looking up a topic's meta data like shard before failing a request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metadata_dissemination_retry_delay_ms:320 - Delay before retry a topic lookup in a shard or other meta tables Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metadata_status_wait_timeout_ms:2000 - Maximum time to wait in metadata request for cluster health to be refreshed Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metrics_reporter_report_interval:86400000 - cluster metrics reporter report interval Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metrics_reporter_tick_interval:60000 - Cluster metrics reporter tick interval Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.metrics_reporter_url:https://m.rp.vectorized.io/v2 - cluster metrics reporter url Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.min_version: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.node_management_operation_timeout_ms:5000 - Timeout for executing node management operations Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.node_status_interval:100 - Time interval between two node status messages. Node status messages establish liveness status outside of the Raft protocol. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.partition_autobalancing_concurrent_moves:50 - Number of partitions that can be reassigned at once Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.partition_autobalancing_max_disk_usage_percent:80 - Disk usage threshold that triggers moving partitions from the node Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.partition_autobalancing_mode:node_add - Partition autobalancing mode Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.partition_autobalancing_movement_batch_size_bytes:5368709120 - Total size of partitions that autobalancer is going to move in one batch Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.partition_autobalancing_node_availability_timeout_sec:900000 - Node unavailability timeout that triggers moving partitions from the node Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.partition_autobalancing_tick_interval_ms:30000 - Partition autobalancer tick interval Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.quota_manager_gc_sec:30000 - Quota manager GC frequency in milliseconds Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_heartbeat_disconnect_failures:3 - After how many failed heartbeats to forcibly close an unresponsive TCP connection. Set to 0 to disable force disconnection. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_heartbeat_interval_ms:150 - Milliseconds for raft leader heartbeats Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_heartbeat_timeout_ms:3000 - raft heartbeat RPC timeout Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_io_timeout_ms:10000 - Raft I/O timeout Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_learner_recovery_rate:104857600 - Raft learner recovery rate limit in bytes per sec Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_max_concurrent_append_requests_per_follower:16 - Maximum number of concurrent append entries requests sent by leader to one follower Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_max_recovery_memory:{nullopt} - Max memory that can be used for reads in raft recovery process by default 15% of total memory Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_recovery_default_read_size:524288 - default size of read issued during raft follower recovery Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_replicate_batch_window_size:1048576 - Max size of requests cached for replication Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_smp_max_non_local_requests:{nullopt} - Maximum number of x-core requests pending in Raft seastar::smp group. (for more details look at `seastar::smp_service_group` documentation) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_timeout_now_timeout_ms:1000 - Timeout for a timeout now request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.raft_transfer_leader_recovery_timeout_ms:10000 - Timeout waiting for follower recovery when transferring leadership Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.readers_cache_eviction_timeout_ms:30000 - Duration after which inactive readers will be evicted from cache Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.reclaim_batch_cache_min_free:67108864 - Free memory limit that will be kept by batch cache background reclaimer Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.reclaim_growth_window:3000 - Length of time in which reclaim sizes grow Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.reclaim_max_size:4194304 - Maximum batch cache reclaim size Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.reclaim_min_size:131072 - Minimum batch cache reclaim size Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.reclaim_stable_window:10000 - Length of time above which growth is reset Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.recovery_append_timeout_ms:5000 - Timeout for append entries requests issued while updating stale follower Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.release_cache_on_segment_roll:0 - Free cache when segments roll Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.replicate_append_timeout_ms:3000 - Timeout for append entries requests issued while replicating entries Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.retention_bytes:{nullopt} - Default max bytes per partition on disk before triggering a compaction Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.retention_local_target_bytes_default:{nullopt} - Local retention size target for partitions of topics with cloud storage write enabled Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.retention_local_target_ms_default:86400000 - Local retention time target for partitions of topics with cloud storage write enabled Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rm_sync_timeout_ms:10000 - Time to wait state catch up before rejecting a request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rm_violation_recovery_policy:crash - Describes how to recover from an invariant violation happened on the partition level Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rpc_server_listen_backlog:{nullopt} - TCP connection queue length for Kafka server and internal RPC server Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rpc_server_tcp_recv_buf:{nullopt} - Internal RPC TCP receive buffer size in bytes. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rpc_server_tcp_send_buf:{nullopt} - Internal RPC TCP transmit buffer size in bytes. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rps_limit_acls_and_users_operations:1000 - Rate limit for controller acls and users operations Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rps_limit_configuration_operations:1000 - Rate limit for controller configuration operations Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rps_limit_move_operations:1000 - Rate limit for controller move operations Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rps_limit_node_management_operations:1000 - Rate limit for controller node management operations Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.rps_limit_topic_operations:1000 - Rate limit for controller topic operations Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.seed_server_meta_topic_partitions: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.segment_appender_flush_timeout_ms:1000 - Maximum delay until buffered data is written Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.segment_fallocation_step:33554432 - Size for segments fallocation Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.seq_table_min_size:1000 - Minimum size of the seq table non affected by compaction Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_compaction_index_memory:134217728 - Maximum number of bytes that may be used on each shard by compactionindex writers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_max_concurrent_replay:1024 - Maximum number of partitions' logs that will be replayed concurrently at startup, or flushed concurrently on shutdown. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_min_free_bytes:5368709120 - Threshold of minimum bytes free space before rejecting producers. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_read_buffer_size:131072 - Size of each read buffer (one per in-flight read, per log segment) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_read_readahead_count:10 - How many additional reads to issue ahead of current read location Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_space_alert_free_threshold_bytes:0 - Threshold of minimim bytes free space before setting storage space alert Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_space_alert_free_threshold_percent:5 - Threshold of minimim percent free space before setting storage space alert Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_strict_data_init:0 - Requires that an empty file named `.redpanda_data_dir` be present in the data directory. Redpanda will refuse to start if it is not found. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.storage_target_replay_bytes:10737418240 - Target bytes to replay from disk on startup after clean shutdown: controls frequency of snapshots and checkpoints Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.superusers:{} - List of superuser usernames Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.target_quota_byte_rate:2147483648 - Target quota byte rate (bytes per second) - 2GB default Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.tm_sync_timeout_ms:10000 - Time to wait state catch up before rejecting a request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.tm_violation_recovery_policy:crash - Describes how to recover from an invariant violation happened on the transaction coordinator level Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.topic_fds_per_partition:{5} - Required file handles per partition when creating topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.topic_memory_per_partition:{1048576} - Required memory per partition when creating topics Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.topic_partitions_per_shard:7000 - Maximum number of partitions which may be allocated to one shard (CPU core) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.topic_partitions_reserve_shard0:2 - Reserved partition slots on shard (CPU core) 0 on each node. If this is >= topic_partitions_per_core, no data partitions will be scheduled on shard 0 Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.transaction_coordinator_cleanup_policy:delete - Cleanup policy for a transaction coordinator topic Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.transaction_coordinator_delete_retention_ms:86400000 - delete segments older than this - default 1 week Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.transaction_coordinator_log_segment_size:1073741824 - How large in bytes should each log segment be (default 1G) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.transaction_coordinator_replication: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.transactional_id_expiration_ms:604800000 - Producer ids are expired once this time has elapsed after the last write with the given producer id. Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.tx_timeout_delay_ms:1000 - Delay before scheduling next check for timed out transactions Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.use_scheduling_groups: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.wait_for_leader_timeout_ms:5000 - Timeout (ms) to wait for leadership in metadata cache Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:515 - redpanda.zstd_decompress_workspace_bytes:8388608 - Size of the zstd decompression workspace Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:543 - Node configuration properties: Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,854 [shard 0] main - application.cc:544 - (use `rpk config set <cfg> <value>` to change) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.admin:{{:{host: 10.16.67.48, port: 9644}}} - Address and port of admin server Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.admin_api_doc_dir:/usr/share/redpanda/admin-api-doc - Admin API doc directory Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.admin_api_tls:{} - TLS configuration for admin HTTP server Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.advertised_kafka_api:{} - Address of Kafka API published to the clients Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.advertised_rpc_api:{nullopt} - Address of RPC endpoint published to other cluster members Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.cloud_storage_cache_directory:{nullopt} - Directory for archival cache. Should be present when `cloud_storage_enabled` is present Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.coproc_supervisor_server:{host: 127.0.0.1, port: 43189} - IpAddress and port for supervisor service Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.dashboard_dir: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.data_directory:{data_directory="/var/lib/redpanda/data"} - Place where redpanda will keep the data Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.developer_mode:0 - Skips most of the checks performed at startup, not recomended for production use Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.empty_seed_starts_cluster:1 - If enabled, requires exactly one node in a cluster-to-be to have its seed_servers list empty. Otherwise, seed_servers list cannot be empty, and must be the same in each node from that list Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.enable_central_config: - Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.kafka_api:{{:{host: 10.16.67.48, port: 9092}:{nullopt}}} - Address and port of an interface to listen for Kafka API requests Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.kafka_api_tls:{} - TLS configuration for Kafka API endpoint Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.node_id:{0} - Unique id identifying a node in the cluster. If missing, a unique id will be assigned for this node when it joins the cluster Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.rack:{nullopt} - Rack identifier Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.rpc_server:{host: 10.16.67.48, port: 33145} - IpAddress and port for RPC server Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.rpc_server_tls:{ enabled: 0 key/cert files: {nullopt} ca file: {nullopt} client_auth_required: 0 } - TLS configuration for RPC server Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - redpanda.seed_servers:{} - List of the seed servers used to join current cluster. If the seed_server list is empty the node will be a cluster root and it will form a new cluster Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.advertised_pandaproxy_api:{} - Rest API address and port to publish to client Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.api_doc_dir:/usr/share/redpanda/proxy-api-doc - API doc directory Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.client_cache_max_size:10 - The maximum number of kafka clients in the LRU cache Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.client_keep_alive:300000 - Time in milliseconds that an idle connection may remain open Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.consumer_instance_timeout_ms:300000 - How long to wait for an idle consumer before removing it Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.pandaproxy_api:{{:{host: 0.0.0.0, port: 8082}:<nullopt>}} - Rest API listen address and port Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy.pandaproxy_api_tls:{} - TLS configuration for Pandaproxy api Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.broker_tls:{ enabled: 0 key/cert files: {nullopt} ca file: {nullopt} client_auth_required: 0 } - TLS configuration for the brokers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.brokers:{{host: 10.16.67.48, port: 9092}} - List of address and port of the brokers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.client_identifier:{test_client} - Identifier to use within the kafka request header Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.consumer_heartbeat_interval_ms:500 - Interval (in milliseconds) for consumer heartbeats Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.consumer_rebalance_timeout_ms:2000 - Timeout (in milliseconds) for consumer rebalance Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.consumer_request_max_bytes:1048576 - Max bytes to fetch per request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.consumer_request_timeout_ms:100 - Interval (in milliseconds) for consumer request timeout Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.consumer_session_timeout_ms:300000 - Timeout (in milliseconds) for consumer session Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.produce_batch_delay_ms:100 - Delay (in milliseconds) to wait before sending batch Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.produce_batch_record_count:1000 - Number of records to batch before sending to broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.produce_batch_size_bytes:1048576 - Number of bytes to batch before sending to broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.retries:5 - Number of times to retry a request to a broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.retry_base_backoff_ms:100 - Delay (in milliseconds) for initial retry backoff Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.sasl_mechanism: - The SASL mechanism to use when connecting Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.scram_password: - Password to use for SCRAM authentication mechanisms Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - pandaproxy_client.scram_username: - Username to use for SCRAM authentication mechanisms Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry.api_doc_dir:/usr/share/redpanda/proxy-api-doc - API doc directory Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry.schema_registry_api:{{:{host: 0.0.0.0, port: 8081}:<nullopt>}} - Schema Registry API listen address and port Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry.schema_registry_api_tls:{} - TLS configuration for Schema Registry API Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry.schema_registry_replication_factor:{nullopt} - Replication factor for internal _schemas topic. If unset, defaults to `default_topic_replication` Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.broker_tls:{ enabled: 0 key/cert files: {nullopt} ca file: {nullopt} client_auth_required: 0 } - TLS configuration for the brokers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.brokers:{{host: 10.16.67.48, port: 9092}} - List of address and port of the brokers Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.client_identifier:{test_client} - Identifier to use within the kafka request header Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.consumer_heartbeat_interval_ms:500 - Interval (in milliseconds) for consumer heartbeats Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.consumer_rebalance_timeout_ms:2000 - Timeout (in milliseconds) for consumer rebalance Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.consumer_request_max_bytes:1048576 - Max bytes to fetch per request Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.consumer_request_timeout_ms:100 - Interval (in milliseconds) for consumer request timeout Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.consumer_session_timeout_ms:10000 - Timeout (in milliseconds) for consumer session Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.produce_batch_delay_ms:0 - Delay (in milliseconds) to wait before sending batch Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.produce_batch_record_count:0 - Number of records to batch before sending to broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.produce_batch_size_bytes:0 - Number of bytes to batch before sending to broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.retries:5 - Number of times to retry a request to a broker Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.retry_base_backoff_ms:100 - Delay (in milliseconds) for initial retry backoff Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.sasl_mechanism: - The SASL mechanism to use when connecting Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.scram_password: - Password to use for SCRAM authentication mechanisms Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,855 [shard 0] main - application.cc:515 - schema_registry_client.scram_username: - Username to use for SCRAM authentication mechanisms Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,868 [shard 0] syschecks - Writing pid file "/var/lib/redpanda/data/pid.lock" Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,878 [shard 0] syschecks - STATUS=checking environment (CPU, Mem) Nov 17 08:48:12 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:12,879 [shard 0] storage - directories.h:33 - Checking `/var/lib/redpanda/data` for supported filesystems Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,391 [shard 0] syschecks - STATUS=Constructing storage services Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,392 [shard 0] syschecks - STATUS=Creating feature table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,396 [shard 0] syschecks - STATUS=Constructing internal RPC services {{://10.16.67.48:33145:PLAINTEXT}, max_service_memory_per_core: 783915417, metrics_enabled:true} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,397 [shard 0] syschecks - STATUS=Starting storage services Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 48] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:934, committed_offset:947, dirty_offset:947}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/48_0/934-0-v1.log, (2278 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/48_0/934-0-v1.base_index, offsets:{934}, index:{header_bitflags:0, base_offset:{934}, max_offset:{947}, base_timestamp:{timestamp: 1668633097517}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 6] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1441, committed_offset:1450, dirty_offset:1450}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/6_0/1441-0-v1.log, (2738 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/6_0/1441-0-v1.base_index, offsets:{1441}, index:{header_bitflags:0, base_offset:{1441}, max_offset:{1450}, base_timestamp:{timestamp: 1668633097528}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 48] kvstore - kvstore.cc:509 - Replaying segment with base offset 934 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 11] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1160, committed_offset:1186, dirty_offset:1186}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/11_0/1160-0-v1.log, (9169 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/11_0/1160-0-v1.base_index, offsets:{1160}, index:{header_bitflags:0, base_offset:{1160}, max_offset:{1186}, base_timestamp:{timestamp: 1668633097526}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 11] kvstore - kvstore.cc:509 - Replaying segment with base offset 1160 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 63] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:63073, committed_offset:63541, dirty_offset:63541}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/63_0/63073-0-v1.log, (71355 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/63_0/63073-0-v1.base_index, offsets:{63073}, index:{header_bitflags:0, base_offset:{63073}, max_offset:{63541}, base_timestamp:{timestamp: 1668633097536}, max_timestamp:{timestamp: 1668692181805}, index(3,3,3)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 45] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:14267, committed_offset:14449, dirty_offset:14449}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/45_0/14267-0-v1.log, (24302 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/45_0/14267-0-v1.base_index, offsets:{14267}, index:{header_bitflags:0, base_offset:{14267}, max_offset:{14449}, base_timestamp:{timestamp: 1668633097361}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 6] kvstore - kvstore.cc:509 - Replaying segment with base offset 1441 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 63] kvstore - kvstore.cc:509 - Replaying segment with base offset 63073 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 45] kvstore - kvstore.cc:509 - Replaying segment with base offset 14267 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 10] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1216, committed_offset:1248, dirty_offset:1248}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/10_0/1216-0-v1.log, (9936 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/10_0/1216-0-v1.base_index, offsets:{1216}, index:{header_bitflags:0, base_offset:{1216}, max_offset:{1248}, base_timestamp:{timestamp: 1668633097520}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 10] kvstore - kvstore.cc:509 - Replaying segment with base offset 1216 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 36] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:4928, committed_offset:5020, dirty_offset:5020}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/36_0/4928-0-v1.log, (18447 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/36_0/4928-0-v1.base_index, offsets:{4928}, index:{header_bitflags:0, base_offset:{4928}, max_offset:{5020}, base_timestamp:{timestamp: 1668633097509}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 36] kvstore - kvstore.cc:509 - Replaying segment with base offset 4928 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 7] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:340417, committed_offset:341733, dirty_offset:341733}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/7_0/340417-0-v1.log, (293768 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/7_0/340417-0-v1.base_index, offsets:{340417}, index:{header_bitflags:0, base_offset:{340417}, max_offset:{341733}, base_timestamp:{timestamp: 1668633097540}, max_timestamp:{timestamp: 1668692181805}, index(9,9,9)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 7] kvstore - kvstore.cc:509 - Replaying segment with base offset 340417 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 22] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1186, committed_offset:1221, dirty_offset:1221}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/22_0/1186-0-v1.log, (18273 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/22_0/1186-0-v1.base_index, offsets:{1186}, index:{header_bitflags:0, base_offset:{1186}, max_offset:{1221}, base_timestamp:{timestamp: 1668633097523}, max_timestamp:{timestamp: 1668692181800}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 22] kvstore - kvstore.cc:509 - Replaying segment with base offset 1186 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 53] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:9460, committed_offset:9505, dirty_offset:9505}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/53_0/9460-0-v1.log, (11168 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/53_0/9460-0-v1.base_index, offsets:{9460}, index:{header_bitflags:0, base_offset:{9460}, max_offset:{9505}, base_timestamp:{timestamp: 1668633097521}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 53] kvstore - kvstore.cc:509 - Replaying segment with base offset 9460 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 2] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:11509, committed_offset:11613, dirty_offset:11613}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/2_0/11509-0-v1.log, (19839 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/2_0/11509-0-v1.base_index, offsets:{11509}, index:{header_bitflags:0, base_offset:{11509}, max_offset:{11613}, base_timestamp:{timestamp: 1668633097553}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 30] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:13660, committed_offset:13820, dirty_offset:13820}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/30_0/13660-0-v1.log, (29985 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/30_0/13660-0-v1.base_index, offsets:{13660}, index:{header_bitflags:0, base_offset:{13660}, max_offset:{13820}, base_timestamp:{timestamp: 1668633097516}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 12] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1869, committed_offset:1909, dirty_offset:1909}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/12_0/1869-0-v1.log, (11975 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/12_0/1869-0-v1.base_index, offsets:{1869}, index:{header_bitflags:0, base_offset:{1869}, max_offset:{1909}, base_timestamp:{timestamp: 1668633097520}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,414 [shard 28] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:403919, committed_offset:406327, dirty_offset:406327}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/28_0/403919-0-v1.log, (436949 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/28_0/403919-0-v1.base_index, offsets:{403919}, index:{header_bitflags:0, base_offset:{403919}, max_offset:{406327}, base_timestamp:{timestamp: 1668633097517}, max_timestamp:{timestamp: 1668692181805}, index(14,14,14)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 37] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:5864, committed_offset:5942, dirty_offset:5942}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/37_0/5864-0-v1.log, (11201 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/37_0/5864-0-v1.base_index, offsets:{5864}, index:{header_bitflags:0, base_offset:{5864}, max_offset:{5942}, base_timestamp:{timestamp: 1668633097510}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 40] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:115718, committed_offset:116858, dirty_offset:116858}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/40_0/115718-0-v1.log, (179547 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/40_0/115718-0-v1.base_index, offsets:{115718}, index:{header_bitflags:0, base_offset:{115718}, max_offset:{116858}, base_timestamp:{timestamp: 1668633097599}, max_timestamp:{timestamp: 1668692181805}, index(6,6,6)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 52] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:9394, committed_offset:9427, dirty_offset:9427}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/52_0/9394-0-v1.log, (5086 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/52_0/9394-0-v1.base_index, offsets:{9394}, index:{header_bitflags:0, base_offset:{9394}, max_offset:{9427}, base_timestamp:{timestamp: 1668633097522}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 49] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:28764, committed_offset:28963, dirty_offset:28963}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/49_0/28764-0-v1.log, (26005 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/49_0/28764-0-v1.base_index, offsets:{28764}, index:{header_bitflags:0, base_offset:{28764}, max_offset:{28963}, base_timestamp:{timestamp: 1668633097497}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 14] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:130180, committed_offset:131245, dirty_offset:131245}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/14_0/130180-0-v1.log, (208404 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/14_0/130180-0-v1.base_index, offsets:{130180}, index:{header_bitflags:0, base_offset:{130180}, max_offset:{131245}, base_timestamp:{timestamp: 1668633097516}, max_timestamp:{timestamp: 1668692181805}, index(7,7,7)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 13] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:132701, committed_offset:133773, dirty_offset:133773}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/13_0/132701-0-v1.log, (206385 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/13_0/132701-0-v1.base_index, offsets:{132701}, index:{header_bitflags:0, base_offset:{132701}, max_offset:{133773}, base_timestamp:{timestamp: 1668633097520}, max_timestamp:{timestamp: 1668692181805}, index(7,7,7)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 35] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:5080, committed_offset:5173, dirty_offset:5173}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/35_0/5080-0-v1.log, (18813 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/35_0/5080-0-v1.base_index, offsets:{5080}, index:{header_bitflags:0, base_offset:{5080}, max_offset:{5173}, base_timestamp:{timestamp: 1668633097518}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 32] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1301, committed_offset:1329, dirty_offset:1329}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/32_0/1301-0-v1.log, (8710 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/32_0/1301-0-v1.base_index, offsets:{1301}, index:{header_bitflags:0, base_offset:{1301}, max_offset:{1329}, base_timestamp:{timestamp: 1668633097522}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 4] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:164917, committed_offset:165989, dirty_offset:165989}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/4_0/164917-0-v1.log, (137899 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/4_0/164917-0-v1.base_index, offsets:{164917}, index:{header_bitflags:0, base_offset:{164917}, max_offset:{165989}, base_timestamp:{timestamp: 1668633097521}, max_timestamp:{timestamp: 1668692181805}, index(5,5,5)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 39] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:5763, committed_offset:5828, dirty_offset:5828}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/39_0/5763-0-v1.log, (9051 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/39_0/5763-0-v1.base_index, offsets:{5763}, index:{header_bitflags:0, base_offset:{5763}, max_offset:{5828}, base_timestamp:{timestamp: 1668633097518}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 3] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:166527, committed_offset:167602, dirty_offset:167602}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/3_0/166527-0-v1.log, (163343 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/3_0/166527-0-v1.base_index, offsets:{166527}, index:{header_bitflags:0, base_offset:{166527}, max_offset:{167602}, base_timestamp:{timestamp: 1668633097635}, max_timestamp:{timestamp: 1668692181805}, index(5,5,5)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 18] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:6309, committed_offset:6371, dirty_offset:6371}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/18_0/6309-0-v1.log, (15881 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/18_0/6309-0-v1.base_index, offsets:{6309}, index:{header_bitflags:0, base_offset:{6309}, max_offset:{6371}, base_timestamp:{timestamp: 1668633097529}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 21] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1126, committed_offset:1136, dirty_offset:1136}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/21_0/1126-0-v1.log, (6292 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/21_0/1126-0-v1.base_index, offsets:{1126}, index:{header_bitflags:0, base_offset:{1126}, max_offset:{1136}, base_timestamp:{timestamp: 1668633097524}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 24] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:17383, committed_offset:17567, dirty_offset:17567}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/24_0/17383-0-v1.log, (49216 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/24_0/17383-0-v1.base_index, offsets:{17383}, index:{header_bitflags:0, base_offset:{17383}, max_offset:{17567}, base_timestamp:{timestamp: 1668633097519}, max_timestamp:{timestamp: 1668692181792}, index(2,2,2)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 5] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:147356, committed_offset:148314, dirty_offset:148314}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/5_0/147356-0-v1.log, (125244 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/5_0/147356-0-v1.base_index, offsets:{147356}, index:{header_bitflags:0, base_offset:{147356}, max_offset:{148314}, base_timestamp:{timestamp: 1668633097533}, max_timestamp:{timestamp: 1668692181805}, index(4,4,4)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 24] kvstore - kvstore.cc:509 - Replaying segment with base offset 17383 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 31] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1273, committed_offset:1295, dirty_offset:1295}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/31_0/1273-0-v1.log, (4080 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/31_0/1273-0-v1.base_index, offsets:{1273}, index:{header_bitflags:0, base_offset:{1273}, max_offset:{1295}, base_timestamp:{timestamp: 1668633097524}, max_timestamp:{timestamp: 1668692181792}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 8] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:333444, committed_offset:334808, dirty_offset:334808}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/8_0/333444-0-v1.log, (304588 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/8_0/333444-0-v1.base_index, offsets:{333444}, index:{header_bitflags:0, base_offset:{333444}, max_offset:{334808}, base_timestamp:{timestamp: 1668633097519}, max_timestamp:{timestamp: 1668692181805}, index(10,10,10)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 52] kvstore - kvstore.cc:509 - Replaying segment with base offset 9394 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 15] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:127338, committed_offset:128472, dirty_offset:128472}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/15_0/127338-0-v1.log, (220598 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/15_0/127338-0-v1.base_index, offsets:{127338}, index:{header_bitflags:0, base_offset:{127338}, max_offset:{128472}, base_timestamp:{timestamp: 1668633097525}, max_timestamp:{timestamp: 1668692181805}, index(7,7,7)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 59] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:538841, committed_offset:542189, dirty_offset:542189}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/59_0/538841-0-v1.log, (424843 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/59_0/538841-0-v1.base_index, offsets:{538841}, index:{header_bitflags:0, base_offset:{538841}, max_offset:{542189}, base_timestamp:{timestamp: 1668633097511}, max_timestamp:{timestamp: 1668692181800}, index(13,13,13)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 9] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:338364, committed_offset:339721, dirty_offset:339721}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/9_0/338364-0-v1.log, (261532 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/9_0/338364-0-v1.base_index, offsets:{338364}, index:{header_bitflags:0, base_offset:{338364}, max_offset:{339721}, base_timestamp:{timestamp: 1668633097522}, max_timestamp:{timestamp: 1668692181805}, index(8,8,8)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 38] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:5661, committed_offset:5728, dirty_offset:5728}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/38_0/5661-0-v1.log, (11495 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/38_0/5661-0-v1.base_index, offsets:{5661}, index:{header_bitflags:0, base_offset:{5661}, max_offset:{5728}, base_timestamp:{timestamp: 1668633097510}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 38] kvstore - kvstore.cc:509 - Replaying segment with base offset 5661 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 34] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1361, committed_offset:1389, dirty_offset:1389}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/34_0/1361-0-v1.log, (8649 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/34_0/1361-0-v1.base_index, offsets:{1361}, index:{header_bitflags:0, base_offset:{1361}, max_offset:{1389}, base_timestamp:{timestamp: 1668633097521}, max_timestamp:{timestamp: 1668692181792}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 34] kvstore - kvstore.cc:509 - Replaying segment with base offset 1361 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 13] kvstore - kvstore.cc:509 - Replaying segment with base offset 132701 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 35] kvstore - kvstore.cc:509 - Replaying segment with base offset 5080 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 30] kvstore - kvstore.cc:509 - Replaying segment with base offset 13660 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 50] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:28889, committed_offset:29086, dirty_offset:29086}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/50_0/28889-0-v1.log, (31102 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/50_0/28889-0-v1.base_index, offsets:{28889}, index:{header_bitflags:0, base_offset:{28889}, max_offset:{29086}, base_timestamp:{timestamp: 1668633097523}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 17] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:6573, committed_offset:6643, dirty_offset:6643}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/17_0/6573-0-v1.log, (14437 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/17_0/6573-0-v1.base_index, offsets:{6573}, index:{header_bitflags:0, base_offset:{6573}, max_offset:{6643}, base_timestamp:{timestamp: 1668633097577}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 62] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:62551, committed_offset:63017, dirty_offset:63017}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/62_0/62551-0-v1.log, (77503 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/62_0/62551-0-v1.base_index, offsets:{62551}, index:{header_bitflags:0, base_offset:{62551}, max_offset:{63017}, base_timestamp:{timestamp: 1668633097623}, max_timestamp:{timestamp: 1668692181805}, index(3,3,3)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 17] kvstore - kvstore.cc:509 - Replaying segment with base offset 6573 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 62] kvstore - kvstore.cc:509 - Replaying segment with base offset 62551 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 16] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:129061, committed_offset:130171, dirty_offset:130171}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/16_0/129061-0-v1.log, (248585 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/16_0/129061-0-v1.base_index, offsets:{129061}, index:{header_bitflags:0, base_offset:{129061}, max_offset:{130171}, base_timestamp:{timestamp: 1668633097526}, max_timestamp:{timestamp: 1668692181805}, index(8,8,8)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 19] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1365, committed_offset:1382, dirty_offset:1382}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/19_0/1365-0-v1.log, (6991 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/19_0/1365-0-v1.base_index, offsets:{1365}, index:{header_bitflags:0, base_offset:{1365}, max_offset:{1382}, base_timestamp:{timestamp: 1668633097529}, max_timestamp:{timestamp: 1668692181806}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 16] kvstore - kvstore.cc:509 - Replaying segment with base offset 129061 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 46] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:968, committed_offset:981, dirty_offset:981}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/46_0/968-0-v1.log, (1949 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/46_0/968-0-v1.base_index, offsets:{968}, index:{header_bitflags:0, base_offset:{968}, max_offset:{981}, base_timestamp:{timestamp: 1668633097498}, max_timestamp:{timestamp: 1668692181798}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 39] kvstore - kvstore.cc:509 - Replaying segment with base offset 5763 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 18] kvstore - kvstore.cc:509 - Replaying segment with base offset 6309 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 21] kvstore - kvstore.cc:509 - Replaying segment with base offset 1126 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 54] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:9548, committed_offset:9581, dirty_offset:9581}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/54_0/9548-0-v1.log, (5305 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/54_0/9548-0-v1.base_index, offsets:{9548}, index:{header_bitflags:0, base_offset:{9548}, max_offset:{9581}, base_timestamp:{timestamp: 1668633097509}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 51] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:28689, committed_offset:28913, dirty_offset:28913}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/51_0/28689-0-v1.log, (34667 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/51_0/28689-0-v1.base_index, offsets:{28689}, index:{header_bitflags:0, base_offset:{28689}, max_offset:{28913}, base_timestamp:{timestamp: 1668633097516}, max_timestamp:{timestamp: 1668692181800}, index(2,2,2)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 54] kvstore - kvstore.cc:509 - Replaying segment with base offset 9548 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 51] kvstore - kvstore.cc:509 - Replaying segment with base offset 28689 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 40] kvstore - kvstore.cc:509 - Replaying segment with base offset 115718 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 31] kvstore - kvstore.cc:509 - Replaying segment with base offset 1273 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 2] kvstore - kvstore.cc:509 - Replaying segment with base offset 11509 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 5] kvstore - kvstore.cc:509 - Replaying segment with base offset 147356 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 47] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:998, committed_offset:1013, dirty_offset:1013}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/47_0/998-0-v1.log, (2433 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/47_0/998-0-v1.base_index, offsets:{998}, index:{header_bitflags:0, base_offset:{998}, max_offset:{1013}, base_timestamp:{timestamp: 1668633097524}, max_timestamp:{timestamp: 1668692181800}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 15] kvstore - kvstore.cc:509 - Replaying segment with base offset 127338 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 59] kvstore - kvstore.cc:509 - Replaying segment with base offset 538841 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 9] kvstore - kvstore.cc:509 - Replaying segment with base offset 338364 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 60] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:526391, committed_offset:529696, dirty_offset:529696}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/60_0/526391-0-v1.log, (418214 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/60_0/526391-0-v1.base_index, offsets:{526391}, index:{header_bitflags:0, base_offset:{526391}, max_offset:{529696}, base_timestamp:{timestamp: 1668633097513}, max_timestamp:{timestamp: 1668692181805}, index(13,13,13)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 60] kvstore - kvstore.cc:509 - Replaying segment with base offset 526391 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 43] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:24839, committed_offset:25032, dirty_offset:25032}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/43_0/24839-0-v1.log, (25688 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/43_0/24839-0-v1.base_index, offsets:{24839}, index:{header_bitflags:0, base_offset:{24839}, max_offset:{25032}, base_timestamp:{timestamp: 1668633097521}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 14] kvstore - kvstore.cc:509 - Replaying segment with base offset 130180 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 61] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:60778, committed_offset:61220, dirty_offset:61220}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/61_0/60778-0-v1.log, (67143 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/61_0/60778-0-v1.base_index, offsets:{60778}, index:{header_bitflags:0, base_offset:{60778}, max_offset:{61220}, base_timestamp:{timestamp: 1668633097624}, max_timestamp:{timestamp: 1668692181798}, index(3,3,3)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 41] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:115911, committed_offset:117049, dirty_offset:117049}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/41_0/115911-0-v1.log, (173052 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/41_0/115911-0-v1.base_index, offsets:{115911}, index:{header_bitflags:0, base_offset:{115911}, max_offset:{117049}, base_timestamp:{timestamp: 1668633097530}, max_timestamp:{timestamp: 1668692181800}, index(6,6,6)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 61] kvstore - kvstore.cc:509 - Replaying segment with base offset 60778 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 32] kvstore - kvstore.cc:509 - Replaying segment with base offset 1301 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 50] kvstore - kvstore.cc:509 - Replaying segment with base offset 28889 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 1] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:11422, committed_offset:11533, dirty_offset:11533}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/1_0/11422-0-v1.log, (25074 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/1_0/11422-0-v1.base_index, offsets:{11422}, index:{header_bitflags:0, base_offset:{11422}, max_offset:{11533}, base_timestamp:{timestamp: 1668633097537}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 23] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:17487, committed_offset:17664, dirty_offset:17664}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/23_0/17487-0-v1.log, (47657 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/23_0/17487-0-v1.base_index, offsets:{17487}, index:{header_bitflags:0, base_offset:{17487}, max_offset:{17664}, base_timestamp:{timestamp: 1668633097534}, max_timestamp:{timestamp: 1668692181793}, index(2,2,2)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 1] kvstore - kvstore.cc:509 - Replaying segment with base offset 11422 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 23] kvstore - kvstore.cc:509 - Replaying segment with base offset 17487 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 19] kvstore - kvstore.cc:509 - Replaying segment with base offset 1365 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 20] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1106, committed_offset:1138, dirty_offset:1138}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/20_0/1106-0-v1.log, (15331 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/20_0/1106-0-v1.base_index, offsets:{1106}, index:{header_bitflags:0, base_offset:{1106}, max_offset:{1138}, base_timestamp:{timestamp: 1668633097533}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 20] kvstore - kvstore.cc:509 - Replaying segment with base offset 1106 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 3] kvstore - kvstore.cc:509 - Replaying segment with base offset 166527 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 44] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:14202, committed_offset:14384, dirty_offset:14384}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/44_0/14202-0-v1.log, (24363 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/44_0/14202-0-v1.base_index, offsets:{14202}, index:{header_bitflags:0, base_offset:{14202}, max_offset:{14384}, base_timestamp:{timestamp: 1668633097510}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 12] kvstore - kvstore.cc:509 - Replaying segment with base offset 1869 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 44] kvstore - kvstore.cc:509 - Replaying segment with base offset 14202 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 47] kvstore - kvstore.cc:509 - Replaying segment with base offset 998 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 49] kvstore - kvstore.cc:509 - Replaying segment with base offset 28764 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 42] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:114285, committed_offset:115429, dirty_offset:115429}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/42_0/114285-0-v1.log, (145031 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/42_0/114285-0-v1.base_index, offsets:{114285}, index:{header_bitflags:0, base_offset:{114285}, max_offset:{115429}, base_timestamp:{timestamp: 1668633097534}, max_timestamp:{timestamp: 1668692181805}, index(5,5,5)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 28] kvstore - kvstore.cc:509 - Replaying segment with base offset 403919 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 42] kvstore - kvstore.cc:509 - Replaying segment with base offset 114285 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 46] kvstore - kvstore.cc:509 - Replaying segment with base offset 968 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 8] kvstore - kvstore.cc:509 - Replaying segment with base offset 333444 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 37] kvstore - kvstore.cc:509 - Replaying segment with base offset 5864 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 43] kvstore - kvstore.cc:509 - Replaying segment with base offset 24839 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 41] kvstore - kvstore.cc:509 - Replaying segment with base offset 115911 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 4] kvstore - kvstore.cc:509 - Replaying segment with base offset 164917 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 29] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:13838, committed_offset:13999, dirty_offset:13999}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/29_0/13838-0-v1.log, (29248 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/29_0/13838-0-v1.base_index, offsets:{13838}, index:{header_bitflags:0, base_offset:{13838}, max_offset:{13999}, base_timestamp:{timestamp: 1668633097514}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 29] kvstore - kvstore.cc:509 - Replaying segment with base offset 13838 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 25] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:16833, committed_offset:17015, dirty_offset:17015}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/25_0/16833-0-v1.log, (30146 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/25_0/16833-0-v1.base_index, offsets:{16833}, index:{header_bitflags:0, base_offset:{16833}, max_offset:{17015}, base_timestamp:{timestamp: 1668633097523}, max_timestamp:{timestamp: 1668692181805}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,415 [shard 33] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:1339, committed_offset:1367, dirty_offset:1367}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/33_0/1339-0-v1.log, (8856 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/33_0/1339-0-v1.base_index, offsets:{1339}, index:{header_bitflags:0, base_offset:{1339}, max_offset:{1367}, base_timestamp:{timestamp: 1668633097521}, max_timestamp:{timestamp: 1668692181792}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,416 [shard 33] kvstore - kvstore.cc:509 - Replaying segment with base offset 1339 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,416 [shard 25] kvstore - kvstore.cc:509 - Replaying segment with base offset 16833 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,416 [shard 27] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:812142, committed_offset:819026, dirty_offset:819026}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/27_0/812142-0-v1.log, (1229621 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/27_0/812142-0-v1.base_index, offsets:{812142}, index:{header_bitflags:0, base_offset:{812142}, max_offset:{819026}, base_timestamp:{timestamp: 1668633097515}, max_timestamp:{timestamp: 1668692181805}, index(38,38,38)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,416 [shard 26] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:812739, committed_offset:819469, dirty_offset:819469}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/26_0/812739-0-v1.log, (1409773 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/26_0/812739-0-v1.base_index, offsets:{812739}, index:{header_bitflags:0, base_offset:{812739}, max_offset:{819469}, base_timestamp:{timestamp: 1668633097525}, max_timestamp:{timestamp: 1668692181805}, index(43,43,43)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,416 [shard 26] kvstore - kvstore.cc:509 - Replaying segment with base offset 812739 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,416 [shard 27] kvstore - kvstore.cc:509 - Replaying segment with base offset 812142 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 56] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:551824, committed_offset:560024, dirty_offset:560024}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/56_0/551824-0-v1.log, (1049725 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/56_0/551824-0-v1.base_index, offsets:{551824}, index:{header_bitflags:0, base_offset:{551824}, max_offset:{560024}, base_timestamp:{timestamp: 1668633097574}, max_timestamp:{timestamp: 1668692181805}, index(32,32,32)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 57] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:542525, committed_offset:550499, dirty_offset:550499}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/57_0/542525-0-v1.log, (1021997 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/57_0/542525-0-v1.base_index, offsets:{542525}, index:{header_bitflags:0, base_offset:{542525}, max_offset:{550499}, base_timestamp:{timestamp: 1668633097591}, max_timestamp:{timestamp: 1668692181805}, index(32,32,32)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 57] kvstore - kvstore.cc:509 - Replaying segment with base offset 542525 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 56] kvstore - kvstore.cc:509 - Replaying segment with base offset 551824 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 58] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:535757, committed_offset:543667, dirty_offset:543667}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/58_0/535757-0-v1.log, (1011380 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/58_0/535757-0-v1.base_index, offsets:{535757}, index:{header_bitflags:0, base_offset:{535757}, max_offset:{543667}, base_timestamp:{timestamp: 1668633097519}, max_timestamp:{timestamp: 1668692181809}, index(31,31,31)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 58] kvstore - kvstore.cc:509 - Replaying segment with base offset 535757 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 55] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:558318, committed_offset:566744, dirty_offset:566744}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/55_0/558318-0-v1.log, (1358889 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/55_0/558318-0-v1.base_index, offsets:{558318}, index:{header_bitflags:0, base_offset:{558318}, max_offset:{566744}, base_timestamp:{timestamp: 1668633097520}, max_timestamp:{timestamp: 1668692181805}, index(42,42,42)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,417 [shard 55] kvstore - kvstore.cc:509 - Replaying segment with base offset 558318 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,435 [shard 0] storage - segment_set.cc:313 - Recovered: {offset_tracker:{term:0, base_offset:2581, committed_offset:2630, dirty_offset:2630}, compacted_segment=0, finished_self_compaction=0, generation={1}, reader={/var/lib/redpanda/data/redpanda/kvstore/0_0/2581-0-v1.log, (31384 bytes)}, writer=nullptr, cache=nullptr, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/redpanda/kvstore/0_0/2581-0-v1.base_index, offsets:{2581}, index:{header_bitflags:0, base_offset:{2581}, max_offset:{2630}, base_timestamp:{timestamp: 1668633093922}, max_timestamp:{timestamp: 1668692181797}, index(1,1,1)}, step:32768, needs_persistence:0}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,435 [shard 0] kvstore - kvstore.cc:509 - Replaying segment with base offset 2581 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,435 [shard 0] kvstore - kvstore.cc:538 - Removing old segment with base offset 2579 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,436 [shard 0] main - application.cc:1814 - Upgrade in progress! This binary logical version 7, last feature table snapshot version 5 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,439 [shard 0] main - application.cc:1368 - Loaded stored node ID for node: 0 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,440 [shard 0] main - application.cc:1395 - Loaded existing UUID for node: {1cddf71a-283f-47c5-9169-2b6bce0de366} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,442 [shard 0] syschecks - STATUS=Starting internal RPC bootstrap service Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,450 [shard 0] main - application.cc:1428 - Started RPC server listening at {host: 10.16.67.48, port: 33145} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,452 [shard 0] cluster - cluster_discovery.cc:99 - Controller directory /var/lib/redpanda/data/redpanda/controller/0_0 not empty; assuming existing cluster exists Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,452 [shard 0] main - application.cc:1455 - Starting Redpanda with node_id 0, cluster UUID {nullopt} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,458 [shard 0] syschecks - STATUS=Initializing connection cache Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,459 [shard 0] syschecks - STATUS=Building shard-lookup tables Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,459 [shard 0] syschecks - STATUS=Intializing raft recovery throttle Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,459 [shard 0] syschecks - STATUS=Intializing raft group manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,460 [shard 0] syschecks - STATUS=Adding partition manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,461 [shard 0] main - application.cc:872 - Partition manager started Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,462 [shard 0] syschecks - STATUS=Creating cluster::controller Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,465 [shard 0] syschecks - STATUS=Creating kafka metadata cache Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,465 [shard 0] syschecks - STATUS=Adding kafka quota manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,465 [shard 0] syschecks - STATUS=Creating metadata dissemination service Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,466 [shard 0] syschecks - STATUS=Creating kafka group managers Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,466 [shard 0] syschecks - STATUS=Creating kafka group shard mapper Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,467 [shard 0] syschecks - STATUS=Creating kafka group router Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,467 [shard 0] syschecks - STATUS=Creating id allocator frontend Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,467 [shard 0] syschecks - STATUS=Creating group resource manager frontend Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,467 [shard 0] syschecks - STATUS=Creating partition resource manager frontend Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,468 [shard 0] syschecks - STATUS=Creating tx coordinator frontend Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,469 [shard 0] syschecks - STATUS=Starting kafka RPC {{://10.16.67.48:9092:PLAINTEXT}, max_service_memory_per_core: 1175873126, metrics_enabled:true} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,470 [shard 0] syschecks - STATUS=constructing http server Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,470 [shard 0] syschecks - STATUS=Starting the partition manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,471 [shard 0] syschecks - STATUS=Starting the coproc partition manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,471 [shard 0] syschecks - STATUS=Starting Raft group manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,471 [shard 0] syschecks - STATUS=Starting Kafka group manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,472 [shard 0] syschecks - STATUS=Starting controller Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,481 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1132 - Starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,482 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1419 - Recovered persistent state from kvstore: voted for: {id: {3}, revision: {0}}, term: 20 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,482 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1166 - Starting with voted_for {id: {3}, revision: {0}} term 20 initial_state false Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,485 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1209 - Read bootstrap state: data_seen 36 config_seen 0 eol false commit 1423 term 0 prev_idx 1422 prev_term 0 config_tracker -9223372036854775808 commit_base_tracker 1423 configurations {} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,485 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1210 - Current log offsets: {start_offset:0, committed_offset:1423, committed_offset_term:20, dirty_offset:1423, dirty_offset_term:20, last_term_start_offset:1387} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,485 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1234 - Truncating configurations at 1423 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,495 [shard 0] storage - segment.cc:655 - Creating new segment /var/lib/redpanda/data/redpanda/kvstore/0_0/2631-0-v1.log Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,508 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:1332 - started raft, log offsets: {start_offset:0, committed_offset:1423, committed_offset_term:20, dirty_offset:1423, dirty_offset_term:20, last_term_start_offset:1387}, term: 20, configuration: {current: {voters: {{id: {0}, revision: {0}}, {id: {8}, revision: {0}}, {id: {9}, revision: {0}}, {id: {3}, revision: {0}}, {id: {1}, revision: {0}}, {id: {6}, revision: {0}}, {id: {4}, revision: {0}}, {id: {5}, revision: {0}}, {id: {7}, revision: {0}}, {id: {2}, revision: {0}}}, learners: {}}, old:{nullopt}, revision: 0, update: {nullopt}, version: 4, brokers: {{id: 0, kafka_advertised_listeners: {{:{host: 10.16.67.48, port: 9092}}}, rpc_address: {host: 10.16.67.48, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 8, kafka_advertised_listeners: {{:{host: 10.16.67.47, port: 9092}}}, rpc_address: {host: 10.16.67.47, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 9, kafka_advertised_listeners: {{:{host: 10.16.66.78, port: 9092}}}, rpc_address: {host: 10.16.66.78, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 3, kafka_advertised_listeners: {{:{host: 10.16.67.49, port: 9092}}}, rpc_address: {host: 10.16.67.49, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 1, kafka_advertised_listeners: {{:{host: 10.16.67.50, port: 9092}}}, rpc_address: {host: 10.16.67.50, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 6, kafka_advertised_listeners: {{:{host: 10.16.67.46, port: 9092}}}, rpc_address: {host: 10.16.67.46, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 4, kafka_advertised_listeners: {{:{host: 10.16.67.51, port: 9092}}}, rpc_address: {host: 10.16.67.51, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 5, kafka_advertised_listeners: {{:{host: 10.16.67.44, port: 9092}}}, rpc_address: {host: 10.16.67.44, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 7, kafka_advertised_listeners: {{:{host: 10.16.67.45, port: 9092}}}, rpc_address: {host: 10.16.67.45, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}, {id: 2, kafka_advertised_listeners: {{:{host: 10.16.67.43, port: 9092}}}, rpc_address: {host: 10.16.67.43, port: 33145}, rack: {nullopt}, properties: {cores 64, mem_available 233, disk_available 14305}, membership_state: active}}} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 0] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 1] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 8] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 11] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 13] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 15] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 3] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 19] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 24] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 26] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 30] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 34] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 36] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 38] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 4] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 45] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 47] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 52] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 60] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 14] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 18] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 6] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 20] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 2] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 21] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 25] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 22] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 23] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 7] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 27] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 28] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 29] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 10] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 9] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 31] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 32] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 33] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 35] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 37] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 12] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 40] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 41] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 42] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 39] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 43] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 44] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 49] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 46] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 48] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 50] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 51] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 53] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 16] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 56] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 58] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 55] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 54] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 59] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 61] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 57] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 17] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 62] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,513 [shard 63] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,518 [shard 5] cluster - drain_manager.cc:20 - Drain manager starting Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,518 [shard 0] cluster - members_manager.cc:87 - starting cluster::members_manager... Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,519 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,519 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - node_status_backend.cc:61 - Node 8 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,521 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,522 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,522 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,522 [shard 0] cluster - node_status_backend.cc:61 - Node 9 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,523 [shard 0] cluster - node_status_backend.cc:61 - Node 3 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,524 [shard 0] cluster - node_status_backend.cc:61 - Node 1 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,525 [shard 0] cluster - node_status_backend.cc:61 - Node 6 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - node_status_backend.cc:61 - Node 4 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,526 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - node_status_backend.cc:61 - Node 5 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,527 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - node_status_backend.cc:61 - Node 7 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,528 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - node_status_backend.cc:61 - Node 2 has been discovered via members table Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,529 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,530 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,531 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,531 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,539 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,539 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,541 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,541 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,542 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,542 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,542 [shard 0] cluster - members_manager.cc:214 - Applying update to members_manager Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: INFO 2022-11-17 08:48:13,542 [shard 0] cluster - members_manager.cc:216 - Raft config update Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: ERROR 2022-11-17 08:48:13,908 [shard 0] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-xfs-builders-i-0219b56622333e416-1/redpanda/redpanda/src/v/cluster/scheduling/allocation_node.cc:81) 'domain_partitions > allocation_capacity{0} && domain_partitions <= _allocated_partitions' Unable to deallocate partition from core 27 in domain 0 at node {node: 8, max_partitions_per_core: 7000, state: active, partition_capacity: 447993, weights: [(2)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(1)(1)(1)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)], allocated: 5({0: 0, -1: 5})} Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: ERROR 2022-11-17 08:48:13,910 [shard 0] assert - Backtrace below: Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: 0x534cdc4 0x4504914 0x40202bd 0x412d744 0x50e9a6f 0x50ed747 0x50eab19 0x500b6b1 0x50097cf 0x1a6067e 0x53ff549 /opt/redpanda/lib/libc.so.6+0x2d58f /opt/redpanda/lib/libc.so.6+0x2d648 0x1a5b464 Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<std::__1::error_code>, auto cluster::topic_updates_dispatcher::apply_update(model::record_batch)::$_0::operator()<std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> > >(std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> >) const::'lambda'(cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>)::operator()(cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>) const::'lambda'(std::__1::error_code), seastar::future<std::__1::error_code> seastar::future<std::__1::error_code>::then_impl_nrvo<auto cluster::topic_updates_dispatcher::apply_update(model::record_batch)::$_0::operator()<std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> > >(std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> >) const::'lambda'(cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>)::operator()(cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>) const::'lambda'(std::__1::error_code), seastar::future<std::__1::error_code> >(std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> >&&)::'lambda'(seastar::internal::promise_base_with_type<std::__1::error_code>&&, auto cluster::topic_updates_dispatcher::apply_update(model::record_batch)::$_0::operator()<std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> > >(std::__1::variant<cluster::controller_command<model::topic_namespace, cluster::topic_configuration_assignment, (signed char)0, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)2, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, std::__1::vector<model::broker_shard, std::__1::allocator<model::broker_shard> >, (signed char)3, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::incremental_topic_updates, (signed char)4, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, cluster::create_partitions_configuration_assignment, (signed char)5, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<cluster::non_replicable_topic, signed char, (signed char)6, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::ntp, cluster::cancel_moving_partition_replicas_cmd_data, (signed char)7, (model::record_batch_type)6, (cluster::serde_opts)0>, cluster::controller_command<model::topic_namespace, std::__1::vector<cluster::move_topic_replicas_data, std::__1::allocator<cluster::move_topic_replicas_data> >, (signed char)8, (model::record_batch_type)6, (cluster::serde_opts)1> >) const::'lambda'(cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>)::operator()(cluster::controller_command<model::topic_namespace, model::topic_namespace, (signed char)1, (model::record_batch_type)6, (cluster::serde_opts)0>) const::'lambda'(std::__1::error_code)&, seastar::future_state<std::__1::error_code>&&), std::__1::error_code> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()::operator()()::'lambda'(std::__1::error_code), seastar::future<void> seastar::future<std::__1::error_code>::then_impl_nrvo<raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()::operator()()::'lambda'(std::__1::error_code), seastar::future<void> >(raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()::operator()()::'lambda'(std::__1::error_code)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()::operator()()::'lambda'(std::__1::error_code)&, seastar::future_state<std::__1::error_code>&&), std::__1::error_code> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()>(seastar::gate&, raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()&&)::'lambda'(), false>, seastar::futurize<raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()>(seastar::gate&, raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()&&)::'lambda'(), false> >(seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()>(seastar::gate&, raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()&&)::'lambda'(), false>&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()>(seastar::gate&, raft::mux_state_machine<cluster::topic_updates_dispatcher, cluster::security_manager, cluster::members_manager, cluster::data_policy_manager, cluster::config_manager, cluster::feature_backend, cluster::bootstrap_backend>::apply(model::record_batch)::'lambda'()&&)::'lambda'(), false>&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >, raft::state_machine::batch_applicator::operator()(model::record_batch)::$_5, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > seastar::future<void>::then_impl_nrvo<raft::state_machine::batch_applicator::operator()(model::record_batch)::$_5, seastar::future<seastar::bool_class<seastar::stop_iteration_tag> > >(raft::state_machine::batch_applicator::operator()(model::record_batch)::$_5&&)::'lambda'(seastar::internal::promise_base_with_type<seastar::bool_class<seastar::stop_iteration_tag> >&&, raft::state_machine::batch_applicator::operator()(model::record_batch)::$_5&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::internal::repeater<auto model::record_batch_reader::impl::do_action<raft::state_machine::batch_applicator, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)&&)::'lambda'()> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >, auto model::record_batch_reader::impl::do_action<raft::state_machine::batch_applicator, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)&&)::'lambda0'(), auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&) seastar::future<void>::then_impl_nrvo<auto model::record_batch_reader::impl::do_action<raft::state_machine::batch_applicator, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)&&)::'lambda0'(), seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > > >(raft::state_machine::batch_applicator&&)::'lambda'(seastar::internal::promise_base_with_type<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >&&, auto model::record_batch_reader::impl::do_action<raft::state_machine::batch_applicator, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >, auto model::record_batch_reader::impl::do_consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator&, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >)::'lambda'(raft::state_machine::batch_applicator&)&&)::'lambda0'()&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::internal::do_with_state<std::__1::tuple<raft::state_machine::batch_applicator>, seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > > > Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >, seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >::finally_body<auto model::record_batch_reader::consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) &&::'lambda'(), true>, seastar::futurize<raft::state_machine::batch_applicator>::type seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >::then_wrapped_nrvo<seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >, seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >::finally_body<auto model::record_batch_reader::consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) &&::'lambda'(), true> >(seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >::finally_body<auto model::record_batch_reader::consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) &&::'lambda'(), true>&&)::'lambda'(seastar::internal::promise_base_with_type<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >&&, seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >::finally_body<auto model::record_batch_reader::consume<raft::state_machine::batch_applicator>(raft::state_machine::batch_applicator, std::__1::chrono::time_point<seastar::lowres_clock, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > >) &&::'lambda'(), true>&, seastar::future_state<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >&&), detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > > Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, raft::state_machine::apply()::$_2::operator()(model::record_batch_reader) const::'lambda'(detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> >), seastar::future<void> seastar::future<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >::then_impl_nrvo<raft::state_machine::apply()::$_2::operator()(model::record_batch_reader) const::'lambda'(detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> >), seastar::future<void> >(raft::state_machine::apply()::$_2::operator()(model::record_batch_reader) const::'lambda'(detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> >)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, raft::state_machine::apply()::$_2::operator()(model::record_batch_reader) const::'lambda'(detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> >)&, seastar::future_state<detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > >&&), detail::base_named_type<long, model::model_offset_type, std::__1::integral_constant<bool, true> > > Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_7>(raft::state_machine::apply()::$_7&&)::'lambda'(raft::state_machine::apply()::$_7&&), seastar::futurize<raft::state_machine::apply()::$_7>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_7>(raft::state_machine::apply()::$_7&&)::'lambda'(raft::state_machine::apply()::$_7&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_7>(raft::state_machine::apply()::$_7&&)::'lambda'(raft::state_machine::apply()::$_7&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_7>(raft::state_machine::apply()::$_7&&)::'lambda'(raft::state_machine::apply()::$_7&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_8>(raft::state_machine::apply()::$_8&&)::'lambda'(raft::state_machine::apply()::$_8&&), seastar::futurize<raft::state_machine::apply()::$_8>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_8>(raft::state_machine::apply()::$_8&&)::'lambda'(raft::state_machine::apply()::$_8&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_8>(raft::state_machine::apply()::$_8&&)::'lambda'(raft::state_machine::apply()::$_8&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_8>(raft::state_machine::apply()::$_8&&)::'lambda'(raft::state_machine::apply()::$_8&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_9>(raft::state_machine::apply()::$_9&&)::'lambda'(raft::state_machine::apply()::$_9&&), seastar::futurize<raft::state_machine::apply()::$_9>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_9>(raft::state_machine::apply()::$_9&&)::'lambda'(raft::state_machine::apply()::$_9&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_9>(raft::state_machine::apply()::$_9&&)::'lambda'(raft::state_machine::apply()::$_9&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<raft::state_machine::apply()::$_9>(raft::state_machine::apply()::$_9&&)::'lambda'(raft::state_machine::apply()::$_9&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception<raft::state_machine::apply()::$_10>(raft::state_machine::apply()::$_10&&)::'lambda'(raft::state_machine::apply()::$_10&&), seastar::futurize<raft::state_machine::apply()::$_10>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception<raft::state_machine::apply()::$_10>(raft::state_machine::apply()::$_10&&)::'lambda'(raft::state_machine::apply()::$_10&&)>(seastar::future<void> seastar::future<void>::handle_exception<raft::state_machine::apply()::$_10>(raft::state_machine::apply()::$_10&&)::'lambda'(raft::state_machine::apply()::$_10&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception<raft::state_machine::apply()::$_10>(raft::state_machine::apply()::$_10&&)::'lambda'(raft::state_machine::apply()::$_10&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::internal::do_until_state<raft::state_machine::start()::$_3::operator()() const::'lambda'(), raft::state_machine::start()::$_3::operator()() const::'lambda0'()> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(), false>, seastar::futurize<raft::state_machine::start()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(), false> >(seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(), false>&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void>::finally_body<auto seastar::internal::invoke_func_with_gate<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(), false>&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: -------- Nov 17 08:48:13 srv-01-03-404.iad1.trmr.io rpk[16743]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(seastar::abort_requested_exception const&)>(raft::state_machine::start()::$_3&&)::'lambda'(raft::state_machine::start()::$_3&&), seastar::futurize<raft::state_machine::start()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(seastar::abort_requested_exception const&)>(raft::state_machine::start()::$_3&&)::'lambda'(raft::state_machine::start()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(seastar::abort_requested_exception const&)>(raft::state_machine::start()::$_3&&)::'lambda'(raft::state_machine::start()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<raft::state_machine::start()::$_3>(seastar::gate&, raft::state_machine::start()::$_3&&)::'lambda'(seastar::abort_requested_exception const&)>(raft::state_machine::start()::$_3&&)::'lambda'(raft::state_machine::start()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 17 08:48:14 srv-01-03-404.iad1.trmr.io systemd[1]: redpanda.service: main process exited, code=killed, status=4/ILL Nov 17 08:48:14 srv-01-03-404.iad1.trmr.io systemd[1]: Failed to start Redpanda, the fastest queue in the West.. Nov 17 08:48:14 srv-01-03-404.iad1.trmr.io systemd[1]: Unit redpanda.service entered failed state. Nov 17 08:48:14 srv-01-03-404.iad1.trmr.io systemd[1]: redpanda.service failed. Nov 17 08:48:14 srv-01-03-404.iad1.trmr.io systemd[1]: redpanda.service holdoff time over, scheduling restart. Nov 17 08:48:14 srv-01-03-404.iad1.trmr.io systemd[1]: Stopped Redpanda, the fastest queue in the West..

Also, here are the logs from 22.2.7 from when I issued the rpk cluster maintenance enable 0 --wait command and then stopped redpanda. Not sure if there will be anything of use or not.

Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 0] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 1] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 2] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 8] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 13] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 3] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 26] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 12] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 37] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 15] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 29] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 5] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 57] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 38] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 30] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 58] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 19] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 18] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 47] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 41] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 36] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 53] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 39] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 54] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 6] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 27] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 31] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 40] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 20] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 24] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 33] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 32] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 4] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 23] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 7] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 50] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 43] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 22] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 44] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 42] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 17] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 9] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 59] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 28] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 52] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 55] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 16] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 14] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 25] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 51] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 48] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 49] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 34] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 35] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 46] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 60] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 63] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 45] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 11] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 62] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 56] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 10] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,633 [shard 61] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 21] cluster - members_table.cc:216 - marking node 0 in maintenance state
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 0] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 0] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 0] cluster - drain_manager.cc:256 - Node draining has completed on shard 0
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 1] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 8] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 5] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 4] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 11] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 2] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 16] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 26] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 30] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 33] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 32] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 38] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 42] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 10] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 47] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 50] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 56] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 21] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 59] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 63] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 15] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 11] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 25] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 24] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 29] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 22] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 13] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 27] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 3] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 16] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 28] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 23] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 23] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 36] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 34] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 34] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 23] raft - [group_id:6923, {kafka/rxdata-impnurl/2}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1254}} to {id: {9}, revision: {1254}} in term 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 35] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 40] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 39] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 9] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 9] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 44] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 44] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 45] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 45] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 45] cluster - drain_manager.cc:256 - Node draining has completed on shard 45
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 49] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 8] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 51] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 53] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 52] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 52] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 14] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 14] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 54] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 57] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 57] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 58] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 60] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 58] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 58] cluster - drain_manager.cc:256 - Node draining has completed on shard 58
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 21] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 6] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 59] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 6] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 6] cluster - drain_manager.cc:256 - Node draining has completed on shard 6
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 15] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 25] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1204}} to {id: {3}, revision: {1204}} in term 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 15] raft - [group_id:6689, {kafka/pixelverse_conv_us/8}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1206}} to {id: {5}, revision: {1206}} in term 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 24] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 15] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 25] raft - [group_id:6720, {kafka/rx-aerokeys-userdata/9}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1212}} to {id: {7}, revision: {1212}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 25] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 31] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 28] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 37] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1256}} to {id: {3}, revision: {1256}} in term 13
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 36] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 43] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 30] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 36] raft - [group_id:6968, {kafka/rxdata-outstream_pread_event/7}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1262}} to {id: {7}, revision: {1262}} in term 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 23] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 35] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 9] raft - [group_id:6669, {kafka/pixelverse_conv_diagnostic/8}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1202}} to {id: {4}, revision: {1202}} in term 13
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 32] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 46] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 48] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 12] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 12] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 8] raft - [group_id:6875, {kafka/rxdata-dspbidrequest/4}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1244}} to {id: {5}, revision: {1244}} in term 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 53] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 10] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 47] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 47] cluster - drain_manager.cc:256 - Node draining has completed on shard 47
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 10] raft - [group_id:6672, {kafka/pixelverse_conv_eu/1}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1204}} to {id: {4}, revision: {1204}} in term 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 17] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 1] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 54] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 50] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 57] cluster - drain_manager.cc:256 - Node draining has completed on shard 57
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 19] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 60] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 61] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 62] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 62] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 62] cluster - drain_manager.cc:256 - Node draining has completed on shard 62
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 18] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 63] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 59] cluster - drain_manager.cc:256 - Node draining has completed on shard 59
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 18] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 29] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 11] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 18] raft - [group_id:6696, {kafka/rx-aerokeys-devicedata/5}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1208}} to {id: {9}, revision: {1208}} in term 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 18] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 27] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 27] raft - [group_id:6938, {kafka/rxdata-impserve/7}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1256}} to {id: {5}, revision: {1256}} in term 21
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 27] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 16] cluster - drain_manager.cc:256 - Node draining has completed on shard 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 31] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 26] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 7] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 34] raft - [group_id:7320, {kafka/rx-fc-usw/9}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1343}} to {id: {7}, revision: {1343}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 7] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 39] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 40] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 33] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 30] raft - [group_id:6738, {kafka/rx-cm-post/7}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1216}} to {id: {3}, revision: {1216}} in term 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 36] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 5] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 35] raft - [group_id:6963, {kafka/rxdata-outstream_pread_event/2}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1262}} to {id: {6}, revision: {1262}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 35] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 5] raft - [group_id:6655, {kafka/pixelverse_conv_ap/4}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1200}} to {id: {2}, revision: {1200}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 38] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 4] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 52] cluster - drain_manager.cc:256 - Node draining has completed on shard 52
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 12] raft - [group_id:6889, {kafka/rxdata-dvviewability/8}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1246}} to {id: {9}, revision: {1246}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 10] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 53] raft - [group_id:6814, {kafka/rx-uid-map-batch/3}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1232}} to {id: {4}, revision: {1232}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 14] raft - [group_id:7109, {kafka/summarystats-supplystat/8}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1290}} to {id: {2}, revision: {1290}} in term 23
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 55] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 60] cluster - drain_manager.cc:256 - Node draining has completed on shard 60
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 61] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 56] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 63] cluster - drain_manager.cc:256 - Node draining has completed on shard 63
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 13] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 3] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 31] cluster - drain_manager.cc:256 - Node draining has completed on shard 31
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 29] raft - [group_id:6946, {kafka/rxdata-lurl/5}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1258}} to {id: {7}, revision: {1258}} in term 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 37] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 28] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 43] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 43] cluster - drain_manager.cc:256 - Node draining has completed on shard 43
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 41] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 7] raft - [group_id:6661, {kafka/pixelverse_conv_diagnostic/0}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1202}} to {id: {9}, revision: {1202}} in term 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 33] raft - [group_id:7314, {kafka/rx-fc-usw/3}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1343}} to {id: {1}, revision: {1343}} in term 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 7] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 9] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 48] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 46] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 32] raft - [group_id:7313, {kafka/rx-fc-usw/2}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1343}} to {id: {3}, revision: {1343}} in term 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 32] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 38] cluster - drain_manager.cc:256 - Node draining has completed on shard 38
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 5] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 42] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 2] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 54] cluster - drain_manager.cc:256 - Node draining has completed on shard 54
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 2] raft - [group_id:6645, {kafka/dmp-cookie-sync/4}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1198}} to {id: {6}, revision: {1198}} in term 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 53] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 19] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 19] cluster - drain_manager.cc:256 - Node draining has completed on shard 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 17] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 56] cluster - drain_manager.cc:256 - Node draining has completed on shard 56
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,639 [shard 22] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 17] raft - [group_id:6694, {kafka/rx-aerokeys-devicedata/3}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1208}} to {id: {8}, revision: {1208}} in term 20
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 24] raft - [group_id:6929, {kafka/rxdata-impnurl/8}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1254}} to {id: {5}, revision: {1254}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 24] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 37] cluster - drain_manager.cc:256 - Node draining has completed on shard 37
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 40] cluster - drain_manager.cc:256 - Node draining has completed on shard 40
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 30] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 44] cluster - drain_manager.cc:256 - Node draining has completed on shard 44
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 46] cluster - drain_manager.cc:256 - Node draining has completed on shard 46
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 48] cluster - drain_manager.cc:256 - Node draining has completed on shard 48
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 49] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 4] raft - [group_id:6650, {kafka/dmp-cookie-sync/9}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1198}} to {id: {2}, revision: {1198}} in term 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 4] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 21] cluster - drain_manager.cc:256 - Node draining has completed on shard 21
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 55] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 26] raft - [group_id:6933, {kafka/rxdata-impserve/2}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1256}} to {id: {5}, revision: {1256}} in term 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 22] raft - [group_id:6712, {kafka/rx-aerokeys-userdata/1}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1212}} to {id: {9}, revision: {1212}} in term 24
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 26] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 51] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 33] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 8] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 42] cluster - drain_manager.cc:256 - Node draining has completed on shard 42
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 14] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 1] raft - [group_id:6852, {kafka/rxdata-bidresponse_extra/1}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1240}} to {id: {2}, revision: {1240}} in term 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 61] cluster - drain_manager.cc:256 - Node draining has completed on shard 61
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 1] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 17] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 34] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 13] raft - [group_id:6681, {kafka/pixelverse_conv_us/0}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1206}} to {id: {2}, revision: {1206}} in term 24
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 7] raft - [group_id:6661, {kafka/pixelverse_conv_diagnostic/0}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 16, dirty offset 6817
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 13] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 22] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 12] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 3] cluster - drain_manager.cc:256 - Node draining has completed on shard 3
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 50] cluster - drain_manager.cc:256 - Node draining has completed on shard 50
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 51] raft - [group_id:6808, {kafka/rx-uid-map/7}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1230}} to {id: {6}, revision: {1230}} in term 23
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 25] raft - [group_id:6720, {kafka/rx-aerokeys-userdata/9}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 39] cluster - drain_manager.cc:256 - Node draining has completed on shard 39
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 49] cluster - drain_manager.cc:256 - Node draining has completed on shard 49
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 55] cluster - drain_manager.cc:256 - Node draining has completed on shard 55
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 27] raft - [group_id:6938, {kafka/rxdata-impserve/7}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 21, dirty offset 2768205137
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 7] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 41] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 24] raft - [group_id:6929, {kafka/rxdata-impnurl/8}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 96728752
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 30] raft - [group_id:6738, {kafka/rx-cm-post/7}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 16, dirty offset 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 29] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 25] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 2] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 27] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 7] cluster - drain_manager.cc:256 - Node draining has completed on shard 7
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 24] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 41] cluster - drain_manager.cc:256 - Node draining has completed on shard 41
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 30] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 24] cluster - drain_manager.cc:256 - Node draining has completed on shard 24
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 30] cluster - drain_manager.cc:256 - Node draining has completed on shard 30
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 1] raft - [group_id:6852, {kafka/rxdata-bidresponse_extra/1}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 17, dirty offset 12342762
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 27] cluster - drain_manager.cc:256 - Node draining has completed on shard 27
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 22] raft - [group_id:6712, {kafka/rx-aerokeys-userdata/1}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 24, dirty offset 23
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 51] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,640 [shard 25] cluster - drain_manager.cc:256 - Node draining has completed on shard 25
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 13, dirty offset 2768304980
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 1] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 28] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 1] cluster - drain_manager.cc:256 - Node draining has completed on shard 1
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 36] raft - [group_id:6968, {kafka/rxdata-outstream_pread_event/7}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 15, dirty offset 44804
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 5] raft - [group_id:6655, {kafka/pixelverse_conv_ap/4}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 22] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 14] raft - [group_id:7109, {kafka/summarystats-supplystat/8}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 23, dirty offset 6103
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 18] raft - [group_id:6696, {kafka/rx-aerokeys-devicedata/5}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 17, dirty offset 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 29] raft - [group_id:6946, {kafka/rxdata-lurl/5}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 17, dirty offset 433872858
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 16, dirty offset 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 10] raft - [group_id:6672, {kafka/pixelverse_conv_eu/1}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 15, dirty offset 14
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 17] raft - [group_id:6694, {kafka/rx-aerokeys-devicedata/3}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 20, dirty offset 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 5] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 5] cluster - drain_manager.cc:256 - Node draining has completed on shard 5
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 22] cluster - drain_manager.cc:256 - Node draining has completed on shard 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 14] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 29] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 11] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 29] cluster - drain_manager.cc:256 - Node draining has completed on shard 29
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 35] raft - [group_id:6963, {kafka/rxdata-outstream_pread_event/2}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 44978
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 32] raft - [group_id:7313, {kafka/rx-fc-usw/2}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 12, dirty offset 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 9] raft - [group_id:6669, {kafka/pixelverse_conv_diagnostic/8}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 13, dirty offset 6788
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 35] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 4] raft - [group_id:6650, {kafka/dmp-cookie-sync/9}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 67526875
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 14] cluster - drain_manager.cc:256 - Node draining has completed on shard 14
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 9] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 33] raft - [group_id:7314, {kafka/rx-fc-usw/3}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 12, dirty offset 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 12] raft - [group_id:6889, {kafka/rxdata-dvviewability/8}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 5587526
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 34] raft - [group_id:7320, {kafka/rx-fc-usw/9}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 10] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 10] cluster - drain_manager.cc:256 - Node draining has completed on shard 10
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 53] raft - [group_id:6814, {kafka/rx-uid-map-batch/3}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 19, dirty offset 12488437
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 26] raft - [group_id:6933, {kafka/rxdata-impserve/2}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 22, dirty offset 2768351744
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 53] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 36] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 36] cluster - drain_manager.cc:256 - Node draining has completed on shard 36
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 8] raft - [group_id:6875, {kafka/rxdata-dspbidrequest/4}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 22, dirty offset 1177612364
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 33] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 34] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 34] cluster - drain_manager.cc:256 - Node draining has completed on shard 34
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 18] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 2] raft - [group_id:6645, {kafka/dmp-cookie-sync/4}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 18, dirty offset 67526307
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 13] raft - [group_id:6681, {kafka/pixelverse_conv_us/0}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 24, dirty offset 1050
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 2] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 2] cluster - drain_manager.cc:256 - Node draining has completed on shard 2
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 13] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 35] cluster - drain_manager.cc:256 - Node draining has completed on shard 35
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 32] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 32] cluster - drain_manager.cc:256 - Node draining has completed on shard 32
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 11] cluster - drain_manager.cc:256 - Node draining has completed on shard 11
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 51] raft - [group_id:6808, {kafka/rx-uid-map/7}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 23, dirty offset 21423199
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 33] cluster - drain_manager.cc:256 - Node draining has completed on shard 33
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 12] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 51] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 12] cluster - drain_manager.cc:256 - Node draining has completed on shard 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 51] cluster - drain_manager.cc:256 - Node draining has completed on shard 51
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 13] cluster - drain_manager.cc:256 - Node draining has completed on shard 13
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 4] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 9] cluster - drain_manager.cc:256 - Node draining has completed on shard 9
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 28] cluster - drain_manager.cc:256 - Node draining has completed on shard 28
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 17] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 26] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 26] cluster - drain_manager.cc:256 - Node draining has completed on shard 26
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 4] cluster - drain_manager.cc:256 - Node draining has completed on shard 4
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 8] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 8] cluster - drain_manager.cc:256 - Node draining has completed on shard 8
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 15] raft - [group_id:6689, {kafka/pixelverse_conv_us/8}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 22, dirty offset 1049
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 17] cluster - drain_manager.cc:256 - Node draining has completed on shard 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 15] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 15] cluster - drain_manager.cc:256 - Node draining has completed on shard 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 18] cluster - drain_manager.cc:256 - Node draining has completed on shard 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 53] cluster - drain_manager.cc:256 - Node draining has completed on shard 53
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 23] raft - [group_id:6923, {kafka/rxdata-impnurl/2}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 18, dirty offset 96754064
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 23] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,641 [shard 23] cluster - drain_manager.cc:256 - Node draining has completed on shard 23
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,650 [shard 7] raft - [group_id:6661, {kafka/pixelverse_conv_diagnostic/0}] consensus.cc:1566 - Received vote request with larger term from node {id: {9}, revision: {1202}}, received 17, current 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,650 [shard 25] raft - [group_id:6720, {kafka/rx-aerokeys-userdata/9}] consensus.cc:1566 - Received vote request with larger term from node {id: {7}, revision: {1212}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,650 [shard 30] raft - [group_id:6738, {kafka/rx-cm-post/7}] consensus.cc:1566 - Received vote request with larger term from node {id: {3}, revision: {1216}}, received 17, current 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,650 [shard 24] raft - [group_id:6929, {kafka/rxdata-impnurl/8}] consensus.cc:1566 - Received vote request with larger term from node {id: {5}, revision: {1254}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] consensus.cc:1566 - Received vote request with larger term from node {id: {3}, revision: {1256}}, received 14, current 13
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 1] raft - [group_id:6852, {kafka/rxdata-bidresponse_extra/1}] consensus.cc:1566 - Received vote request with larger term from node {id: {2}, revision: {1240}}, received 18, current 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 5] raft - [group_id:6655, {kafka/pixelverse_conv_ap/4}] consensus.cc:1566 - Received vote request with larger term from node {id: {2}, revision: {1200}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 14] raft - [group_id:7109, {kafka/summarystats-supplystat/8}] consensus.cc:1566 - Received vote request with larger term from node {id: {2}, revision: {1290}}, received 24, current 23
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 36] raft - [group_id:6968, {kafka/rxdata-outstream_pread_event/7}] consensus.cc:1566 - Received vote request with larger term from node {id: {7}, revision: {1262}}, received 16, current 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] consensus.cc:1566 - Received vote request with larger term from node {id: {3}, revision: {1204}}, received 17, current 16
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 18] raft - [group_id:6696, {kafka/rx-aerokeys-devicedata/5}] consensus.cc:1566 - Received vote request with larger term from node {id: {9}, revision: {1208}}, received 18, current 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 29] raft - [group_id:6946, {kafka/rxdata-lurl/5}] consensus.cc:1566 - Received vote request with larger term from node {id: {7}, revision: {1258}}, received 18, current 17
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 9] raft - [group_id:6669, {kafka/pixelverse_conv_diagnostic/8}] consensus.cc:1566 - Received vote request with larger term from node {id: {4}, revision: {1202}}, received 14, current 13
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 32] raft - [group_id:7313, {kafka/rx-fc-usw/2}] consensus.cc:1566 - Received vote request with larger term from node {id: {3}, revision: {1343}}, received 13, current 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 4] raft - [group_id:6650, {kafka/dmp-cookie-sync/9}] consensus.cc:1566 - Received vote request with larger term from node {id: {2}, revision: {1198}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 17] raft - [group_id:6694, {kafka/rx-aerokeys-devicedata/3}] consensus.cc:1566 - Received vote request with larger term from node {id: {8}, revision: {1208}}, received 21, current 20
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 23] raft - [group_id:6923, {kafka/rxdata-impnurl/2}] consensus.cc:1566 - Received vote request with larger term from node {id: {9}, revision: {1254}}, received 19, current 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 27] raft - [group_id:6938, {kafka/rxdata-impserve/7}] consensus.cc:1566 - Received vote request with larger term from node {id: {5}, revision: {1256}}, received 22, current 21
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 53] raft - [group_id:6814, {kafka/rx-uid-map-batch/3}] consensus.cc:1566 - Received vote request with larger term from node {id: {4}, revision: {1232}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 22] raft - [group_id:6712, {kafka/rx-aerokeys-userdata/1}] consensus.cc:1566 - Received vote request with larger term from node {id: {9}, revision: {1212}}, received 25, current 24
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 15] raft - [group_id:6689, {kafka/pixelverse_conv_us/8}] consensus.cc:1566 - Received vote request with larger term from node {id: {5}, revision: {1206}}, received 23, current 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 13] raft - [group_id:6681, {kafka/pixelverse_conv_us/0}] consensus.cc:1566 - Received vote request with larger term from node {id: {2}, revision: {1206}}, received 25, current 24
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 51] raft - [group_id:6808, {kafka/rx-uid-map/7}] consensus.cc:1566 - Received vote request with larger term from node {id: {6}, revision: {1230}}, received 24, current 23
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 2] raft - [group_id:6645, {kafka/dmp-cookie-sync/4}] consensus.cc:1566 - Received vote request with larger term from node {id: {6}, revision: {1198}}, received 19, current 18
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 8] raft - [group_id:6875, {kafka/rxdata-dspbidrequest/4}] consensus.cc:1566 - Received vote request with larger term from node {id: {5}, revision: {1244}}, received 23, current 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 33] raft - [group_id:7314, {kafka/rx-fc-usw/3}] consensus.cc:1566 - Received vote request with larger term from node {id: {1}, revision: {1343}}, received 13, current 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,651 [shard 26] raft - [group_id:6933, {kafka/rxdata-impserve/2}] consensus.cc:1566 - Received vote request with larger term from node {id: {5}, revision: {1256}}, received 23, current 22
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] cluster - drain_manager.cc:49 - Node draining is starting
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 12] raft - [group_id:6889, {kafka/rxdata-dvviewability/8}] consensus.cc:1566 - Received vote request with larger term from node {id: {9}, revision: {1246}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] cluster - drain_manager.cc:143 - Node draining has started
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] raft - [group_id:6705, {kafka/rx-aerokeys-ipdata/4}] consensus.cc:2935 - Starting leadership transfer from {id: {0}, revision: {1210}} to {id: {9}, revision: {1210}} in term 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] cluster - drain_manager.cc:207 - Draining leadership from 1 groups
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] raft - [group_id:6705, {kafka/rx-aerokeys-ipdata/4}] consensus.cc:187 - [leadership_transfer] Stepping down as leader in term 12, dirty offset 11
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] cluster - drain_manager.cc:239 - Draining leadership from 1 groups 1 succeeded
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,655 [shard 20] cluster - drain_manager.cc:256 - Node draining has completed on shard 20
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,661 [shard 10] raft - [group_id:6672, {kafka/pixelverse_conv_eu/1}] consensus.cc:1566 - Received vote request with larger term from node {id: {4}, revision: {1204}}, received 16, current 15
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,661 [shard 34] raft - [group_id:7320, {kafka/rx-fc-usw/9}] consensus.cc:1566 - Received vote request with larger term from node {id: {7}, revision: {1343}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,661 [shard 35] raft - [group_id:6963, {kafka/rxdata-outstream_pread_event/2}] consensus.cc:1566 - Received vote request with larger term from node {id: {6}, revision: {1262}}, received 20, current 19
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,665 [shard 20] raft - [group_id:6705, {kafka/rx-aerokeys-ipdata/4}] consensus.cc:1566 - Received vote request with larger term from node {id: {9}, revision: {1210}}, received 13, current 12
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,690 [shard 30] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-cm-post/7_1216/16-17-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,700 [shard 24] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impnurl/8_1254/96728753-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,702 [shard 25] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-userdata/9_1212/19-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,705 [shard 9] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_diagnostic/8_1202/6789-14-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,711 [shard 36] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-outstream_pread_event/7_1262/44805-16-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,713 [shard 27] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impserve/7_1256/2768205138-22-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,713 [shard 35] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-outstream_pread_event/2_1262/44979-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,719 [shard 14] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/summarystats-supplystat/8_1290/6104-24-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,729 [shard 11] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_eu/3_1204/16-17-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,729 [shard 26] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impserve/2_1256/2768351745-23-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,735 [shard 17] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-devicedata/3_1208/20-21-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,736 [shard 20] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-ipdata/4_1210/12-13-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,753 [shard 33] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-fc-usw/3_1343/16-13-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,763 [shard 18] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-devicedata/5_1208/17-18-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,769 [shard 51] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-uid-map/7_1230/21423200-24-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,774 [shard 12] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-dvviewability/8_1246/5587527-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,774 [shard 8] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-dspbidrequest/4_1244/1177612365-23-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,778 [shard 4] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/dmp-cookie-sync/9_1198/67526876-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,781 [shard 10] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_eu/1_1204/15-16-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,787 [shard 32] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-fc-usw/2_1343/13-13-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,789 [shard 5] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_ap/4_1200/19-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,791 [shard 15] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_us/8_1206/1050-23-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,793 [shard 2] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/dmp-cookie-sync/4_1198/67526308-19-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,796 [shard 28] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impserve/9_1256/2768304981-14-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,799 [shard 34] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-fc-usw/9_1343/20-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,800 [shard 7] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_diagnostic/0_1202/6818-17-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,820 [shard 22] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-userdata/1_1212/24-25-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,820 [shard 23] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impnurl/2_1254/96754065-19-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,828 [shard 53] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-uid-map-batch/3_1232/12488438-20-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,828 [shard 13] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_us/0_1206/1051-25-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,831 [shard 29] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-lurl/5_1258/433872859-18-v1.log
Nov 17 08:35:46 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:46,858 [shard 1] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-bidresponse_extra/1_1240/12342763-18-v1.log
Nov 17 08:35:47 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:47,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:48 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:48,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:49 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:49,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:50 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:50,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:51 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:51,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:52 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:52,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:53 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:53,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,140 [shard 58] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:17, base_offset:8004665807, committed_offset:8004867551, dirty_offset:8004867551}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-usersync-us/9_1234/8004665807-17-v1.log, (107553213 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-usersync-us/9_1234/8004665807-17-v1.base_index, offsets:{8004665807}, index:{header_bitflags:0, base_offset:{8004665807}, max_offset:{8004867551}, base_timestamp:{timestamp: 1668605699066}, max_timestamp:{timestamp: 1668605746310}, index(486,486,486)}, step:32768, needs_persistence:0}}
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,140 [shard 58] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-usersync-us/9_1234/8004665807-17-v1.log", "/var/lib/redpanda/data/kafka/rx-usersync-us/9_1234/8004665807-17-v1.base_index"}
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,170 [shard 58] offset_translator - ntp: {kafka/rx-usersync-us/9} - offset_translator.cc:277 - prefix_truncate at offset: 8004867551, new state: {base offset/delta: {8004867551}/17, map size: 1, last delta: 17}
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,458 [shard 62] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:14, base_offset:254690063, committed_offset:254800087, dirty_offset:254800087}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rxdata-bidresponse/3_1238/254690063-14-v1.log, (105277585 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rxdata-bidresponse/3_1238/254690063-14-v1.base_index, offsets:{254690063}, index:{header_bitflags:0, base_offset:{254690063}, max_offset:{254800087}, base_timestamp:{timestamp: 1668430983677}, max_timestamp:{timestamp: 1668432947691}, index(3073,3073,3073)}, step:32768, needs_persistence:0}}
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,458 [shard 62] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rxdata-bidresponse/3_1238/254690063-14-v1.log", "/var/lib/redpanda/data/kafka/rxdata-bidresponse/3_1238/254690063-14-v1.base_index"}
Nov 17 08:35:54 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:54,488 [shard 62] offset_translator - ntp: {kafka/rxdata-bidresponse/3} - offset_translator.cc:277 - prefix_truncate at offset: 254800087, new state: {base offset/delta: {254800087}/13, map size: 2, last delta: 14}
Nov 17 08:35:55 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:55,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:56 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:56,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:57 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:57,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:58 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:58,186 [shard 55] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:13, base_offset:8005249365, committed_offset:8005437367, dirty_offset:8005437367}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-usersync-us/0_1234/8005249365-13-v1.log, (100440578 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-usersync-us/0_1234/8005249365-13-v1.base_index, offsets:{8005249365}, index:{header_bitflags:0, base_offset:{8005249365}, max_offset:{8005437367}, base_timestamp:{timestamp: 1668605699424}, max_timestamp:{timestamp: 1668605744327}, index(455,455,455)}, step:32768, needs_persistence:0}}
Nov 17 08:35:58 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:58,186 [shard 55] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-usersync-us/0_1234/8005249365-13-v1.log", "/var/lib/redpanda/data/kafka/rx-usersync-us/0_1234/8005249365-13-v1.base_index"}
Nov 17 08:35:58 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:58,217 [shard 55] offset_translator - ntp: {kafka/rx-usersync-us/0} - offset_translator.cc:277 - prefix_truncate at offset: 8005437367, new state: {base offset/delta: {8005437367}/13, map size: 2, last delta: 14}
Nov 17 08:35:58 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:58,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:35:59 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:35:59,304 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:00 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:00,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:00 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:00,443 [shard 22] raft - [group_id:6712, {kafka/rx-aerokeys-userdata/1}] consensus.cc:1566 - Received vote request with larger term from node {id: {4}, revision: {1212}}, received 26, current 25
Nov 17 08:36:00 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:00,479 [shard 22] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-userdata/1_1212/25-26-v1.log
Nov 17 08:36:01 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:01,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:02 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:02,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:02 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:02,760 [shard 16] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impbeacon/9_1248/749504034-17-v1.log
Nov 17 08:36:03 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:03,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:03 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:03,520 [shard 57] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/6_1234/8321194434-17-v1.log
Nov 17 08:36:04 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:04,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,168 [shard 55] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/0_1234/8319857959-14-v1.log
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,229 [shard 57] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:17, base_offset:8007003919, committed_offset:8007206679, dirty_offset:8007206679}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-usersync-us/6_1234/8007003919-17-v1.log, (108004183 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-usersync-us/6_1234/8007003919-17-v1.base_index, offsets:{8007003919}, index:{header_bitflags:0, base_offset:{8007003919}, max_offset:{8007206679}, base_timestamp:{timestamp: 1668605717659}, max_timestamp:{timestamp: 1668605762096}, index(488,488,488)}, step:32768, needs_persistence:0}}
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,229 [shard 57] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-usersync-us/6_1234/8007003919-17-v1.log", "/var/lib/redpanda/data/kafka/rx-usersync-us/6_1234/8007003919-17-v1.base_index"}
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,259 [shard 57] offset_translator - ntp: {kafka/rx-usersync-us/6} - offset_translator.cc:277 - prefix_truncate at offset: 8007206679, new state: {base offset/delta: {8007206679}/17, map size: 1, last delta: 17}
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,642 [shard 26] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:18, base_offset:18984496149, committed_offset:18985202084, dirty_offset:18985202084}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-bidrequest-stream/4_1214/18984496149-18-v1.log, (109366968 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-bidrequest-stream/4_1214/18984496149-18-v1.base_index, offsets:{18984496149}, index:{header_bitflags:0, base_offset:{18984496149}, max_offset:{18985202084}, base_timestamp:{timestamp: 1668432812000}, max_timestamp:{timestamp: 1668432959000}, index(3297,3297,3297)}, step:32768, needs_persistence:0}}
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,642 [shard 26] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-bidrequest-stream/4_1214/18984496149-18-v1.log", "/var/lib/redpanda/data/kafka/rx-bidrequest-stream/4_1214/18984496149-18-v1.base_index"}
Nov 17 08:36:05 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:05,673 [shard 26] offset_translator - ntp: {kafka/rx-bidrequest-stream/4} - offset_translator.cc:277 - prefix_truncate at offset: 18985202084, new state: {base offset/delta: {18985202084}/18, map size: 2, last delta: 19}
Nov 17 08:36:06 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:06,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:07 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:07,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:08 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:08,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:09 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:09,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:10 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:10,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:11 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:11,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:11 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:11,906 [shard 56] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/3_1234/8320901771-16-v1.log
Nov 17 08:36:12 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:12,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:12 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:12,948 [shard 26] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-bidrequest-stream/4_1214/20396372245-18-v1.log
Nov 17 08:36:13 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:13,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:14 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:14,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:15 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:15,211 [shard 56] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:16, base_offset:8006309648, committed_offset:8006505235, dirty_offset:8006505235}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-usersync-us/3_1234/8006309648-16-v1.log, (104018962 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-usersync-us/3_1234/8006309648-16-v1.base_index, offsets:{8006309648}, index:{header_bitflags:0, base_offset:{8006309648}, max_offset:{8006505235}, base_timestamp:{timestamp: 1668605724143}, max_timestamp:{timestamp: 1668605769574}, index(467,467,467)}, step:32768, needs_persistence:0}}
Nov 17 08:36:15 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:15,211 [shard 56] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-usersync-us/3_1234/8006309648-16-v1.log", "/var/lib/redpanda/data/kafka/rx-usersync-us/3_1234/8006309648-16-v1.base_index"}
Nov 17 08:36:15 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:15,241 [shard 56] offset_translator - ntp: {kafka/rx-usersync-us/3} - offset_translator.cc:277 - prefix_truncate at offset: 8006505235, new state: {base offset/delta: {8006505235}/15, map size: 1, last delta: 15}
Nov 17 08:36:15 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:15,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,218 [shard 27] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:18, base_offset:2570288095, committed_offset:2570420954, dirty_offset:2570420954}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rxdata-impserve/7_1256/2570288095-18-v1.log, (102658275 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rxdata-impserve/7_1256/2570288095-18-v1.base_index, offsets:{2570288095}, index:{header_bitflags:0, base_offset:{2570288095}, max_offset:{2570420954}, base_timestamp:{timestamp: 1668432740789}, max_timestamp:{timestamp: 1668432966363}, index(2979,2979,2979)}, step:32768, needs_persistence:0}}
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,218 [shard 27] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rxdata-impserve/7_1256/2570288095-18-v1.log", "/var/lib/redpanda/data/kafka/rxdata-impserve/7_1256/2570288095-18-v1.base_index"}
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,218 [shard 27] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:12, base_offset:18984669590, committed_offset:18985342161, dirty_offset:18985342161}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-bidrequest-stream/6_1214/18984669590-12-v1.log, (103704564 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-bidrequest-stream/6_1214/18984669590-12-v1.base_index, offsets:{18984669590}, index:{header_bitflags:0, base_offset:{18984669590}, max_offset:{18985342161}, base_timestamp:{timestamp: 1668432827000}, max_timestamp:{timestamp: 1668432966000}, index(3127,3127,3127)}, step:32768, needs_persistence:0}}
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,218 [shard 27] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-bidrequest-stream/6_1214/18984669590-12-v1.log", "/var/lib/redpanda/data/kafka/rx-bidrequest-stream/6_1214/18984669590-12-v1.base_index"}
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,248 [shard 27] offset_translator - ntp: {kafka/rxdata-impserve/7} - offset_translator.cc:277 - prefix_truncate at offset: 2570420954, new state: {base offset/delta: {2570420954}/16, map size: 5, last delta: 20}
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,248 [shard 27] offset_translator - ntp: {kafka/rx-bidrequest-stream/6} - offset_translator.cc:277 - prefix_truncate at offset: 18985342161, new state: {base offset/delta: {18985342161}/12, map size: 2, last delta: 13}
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:16 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:16,781 [shard 4] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-videoevent/2_1284/2365889976-18-v1.log
Nov 17 08:36:17 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:17,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:18 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:18,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:19 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:19,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:20 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:20,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,305 [shard 26] cluster - controller_backend.cc:678 - partition operation {type: update, ntp: {kafka/rx-bidrequest-stream/4}, offset: 1397, new_assignment: { id: 4, group_id: 6725, replicas: {{node_id: 9, shard: 39}, {node_id: 1, shard: 26}, {node_id: 6, shard: 26}} }, previous_replica_set: {{{node_id: 1, shard: 26}, {node_id: 6, shard: 26}, {node_id: 0, shard: 26}}}} result: Current node is not a leader for partition
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io systemd[1]: Stopping Redpanda, the fastest queue in the West....
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[4810]: Command "stop" is deprecated, use "rpk redpanda stop" instead
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,715 [shard 0] redpanda::main - application.cc:250 - Stopping...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,715 [shard 0] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,715 [shard 0] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 2] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 8] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 12] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 14] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 15] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 18] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 26] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 29] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 25] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 36] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 42] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 48] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 45] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 46] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 56] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 61] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 6] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 16] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 17] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 61] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 16] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 19] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 10] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 21] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 27] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 28] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 21] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 9] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 32] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 38] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 4] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 30] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 38] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 22] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 33] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 34] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 41] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 34] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 39] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 13] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 41] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 39] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 35] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 37] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 40] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 37] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 23] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 40] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 49] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 3] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 49] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 42] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 31] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 59] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 31] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 12] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 52] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 47] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 52] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 57] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 47] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 45] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 60] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 62] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 46] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 48] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 60] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 6] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 5] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 17] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 19] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 10] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 27] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 26] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 29] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 9] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 32] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 4] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 24] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 22] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 24] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 25] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 18] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 20] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 13] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 35] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 1] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 43] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 50] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 43] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 44] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 3] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 8] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 54] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 58] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 54] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 53] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 59] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 53] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 63] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 55] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 57] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 62] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 7] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 14] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 5] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 15] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 30] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 33] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 36] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 1] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 23] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 50] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 44] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 58] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 2] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 63] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 56] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 11] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,717 [shard 28] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 20] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 11] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 7] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,718 [shard 55] rpc - server.cc:262 - kafka rpc protocol - Shutting down 425 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,738 [shard 51] rpc - server.cc:253 - kafka rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,738 [shard 51] rpc - server.cc:262 - kafka rpc protocol - Shutting down 426 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,741 [shard 0] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,741 [shard 0] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,741 [shard 4] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,741 [shard 4] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,741 [shard 0] rpc - server.cc:135 - vectorized internal rpc protocol - Disconnected 10.16.67.44:54400 (applying protocol, parse error)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,741 [shard 0] rpc - server.cc:135 - vectorized internal rpc protocol - Disconnected 10.16.67.46:54016 (applying protocol, parse error)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,742 [shard 42] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,742 [shard 42] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,742 [shard 9] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,742 [shard 9] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 34] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 34] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 35] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 35] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 24] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 24] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 63] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 63] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 34] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:59874 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 49] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 5] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 49] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 5] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 61] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 61] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 3 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 18] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,743 [shard 18] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 58] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 58] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 17] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 17] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 29] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 29] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 21] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 21] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 45] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 45] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 20] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 20] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 39] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 39] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 22] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 22] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 56] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,744 [shard 56] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 15] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 25] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 15] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 25] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 6] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 6] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 27] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 27] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 2] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 12] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 2] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 12] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 58] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.51:54138 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 31] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 31] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 26] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 26] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 54] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 57] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 54] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 31] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.50:56543 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 43] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 44] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 43] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 44] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 57] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 60] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 60] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 8] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 31] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.50:56543 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 8] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 37] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 7] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 7] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:54400 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 37] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:54400 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:54400 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 46] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 46] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 28] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 28] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 50] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 45] rpc - simple_protocol.cc:92 - Disconnected 10.16.66.78:63725 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 50] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 32] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 30] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 32] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 41] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 30] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 56] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:59448 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 41] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 19] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 19] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 1] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 1] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 3] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 7] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.51:62279 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 36] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 3] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,745 [shard 36] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 35] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:55779 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 62] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 11] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 62] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 11] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 38] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 7] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.51:62279 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 38] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 2 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 47] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 47] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 59] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 59] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 36] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.50:59876 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 4] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:54596 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 48] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 35] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:55779 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 48] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:54400 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 48] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.47:63664 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 10] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 19] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:59411 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 10] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 36] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.50:59876 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:54400 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 55] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 55] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 15] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:62095 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 38] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.50:60070 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 52] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 52] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 1 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 4] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:54596 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 38] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.45:57254 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 4] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:54596 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 16] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 4] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:54596 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 16] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 4] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:54596 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 40] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 10] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.50:54282 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 4] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:54596 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 62] rpc - simple_protocol.cc:92 - Disconnected 10.16.66.78:54014 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 40] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 56] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:59448 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:54016 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 13] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 13] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 35] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:56739 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 9] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:62281 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 21] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:51221 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 19] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.49:59411 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:54016 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 14] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:54016 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 14] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:54016 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 53] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:54016 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 53] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 4 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 0] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:54016 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 21] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:51221 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 61] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:64189 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 23] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 23] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,746 [shard 8] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:53192 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.66.78:62453 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 6] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.45:54598 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 49] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:57009 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.45:56501 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 49] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:57009 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 49] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:57009 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 12] rpc - simple_protocol.cc:92 - Disconnected 10.16.66.78:53004 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 6] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.45:54598 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.45:56501 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 49] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:57009 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:63221 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:63221 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:63221 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.44:63221 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:60341 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:60341 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:60341 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:60341 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,747 [shard 53] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.43:60341 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,749 [shard 33] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,749 [shard 33] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,759 [shard 51] rpc - server.cc:253 - vectorized internal rpc protocol - Stopping 1 listeners
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,759 [shard 51] rpc - server.cc:262 - vectorized internal rpc protocol - Shutting down 0 connections
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,763 [shard 19] rpc - simple_protocol.cc:92 - Disconnected 10.16.67.46:58067 (stream closed)
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 0] raft - [group_id:0, {redpanda/controller/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 0] raft - [group_id:7061, {kafka/rxdata-usersync/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 3] raft - [group_id:6648, {kafka/dmp-cookie-sync/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 4] raft - [group_id:6860, {kafka/rxdata-bidresponse_extra/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 2] raft - [group_id:7068, {kafka/rxdata-usersync/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 7] raft - [group_id:7084, {kafka/summarystats-dspstat/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 39] raft - [group_id:6978, {kafka/rxdata-providerdmpevent/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 44] raft - [group_id:6994, {kafka/rxdata-rg_application/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 59] raft - [group_id:7045, {kafka/rxdata-testad/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 12] raft - [group_id:6889, {kafka/rxdata-dvviewability/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 26] raft - [group_id:6933, {kafka/rxdata-impserve/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 28] raft - [group_id:6731, {kafka/rx-cm-post/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 32] raft - [group_id:6744, {kafka/rx-cm-tremor/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 46] raft - [group_id:6791, {kafka/rx-confiant-response/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 24] raft - [group_id:6718, {kafka/rx-aerokeys-userdata/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 51] raft - [group_id:7017, {kafka/rxdata-rg_report/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 53] raft - [group_id:6814, {kafka/rx-uid-map-batch/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 11] raft - [group_id:6885, {kafka/rxdata-dvviewability/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 23] raft - [group_id:6715, {kafka/rx-aerokeys-userdata/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 43] raft - [group_id:6993, {kafka/rxdata-rg_application/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 24] raft - [group_id:7285, {kafka/rx-fc-iad/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 25] raft - [group_id:6930, {kafka/rxdata-impnurl/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 7] raft - [group_id:6872, {kafka/rxdata-dspbidrequest/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 24] raft - [group_id:6929, {kafka/rxdata-impnurl/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 60] raft - [group_id:6838, {kafka/rxdata-bidrequest/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 18] raft - [group_id:6696, {kafka/rx-aerokeys-devicedata/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 5] raft - [group_id:6655, {kafka/pixelverse_conv_ap/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 60] raft - [group_id:7048, {kafka/rxdata-testad/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 14] raft - [group_id:6684, {kafka/pixelverse_conv_us/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 59] raft - [group_id:6834, {kafka/rxdata-bidrequest/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 33] raft - [group_id:6747, {kafka/rx-cm-tremor/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 17] raft - [group_id:6903, {kafka/rxdata-impconfirmation/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 35] raft - [group_id:6963, {kafka/rxdata-outstream_pread_event/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 33] raft - [group_id:7314, {kafka/rx-fc-usw/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 42] raft - [group_id:6989, {kafka/rxdata-pubopportunity/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 20] raft - [group_id:6913, {kafka/rxdata-impexpired/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 48] raft - [group_id:6799, {kafka/rx-confiant-response/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 50] raft - [group_id:6804, {kafka/rx-uid-map/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 10] raft - [group_id:7094, {kafka/summarystats-pubdealstat/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 52] raft - [group_id:7023, {kafka/rxdata-segmentusage/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 13] raft - [group_id:6681, {kafka/pixelverse_conv_us/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 57] raft - [group_id:7037, {kafka/rxdata-sspbidresponse/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 61] raft - [group_id:6841, {kafka/rxdata-bidresponse/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 15] raft - [group_id:6689, {kafka/pixelverse_conv_us/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 21] raft - [group_id:6919, {kafka/rxdata-impexpired/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 61] raft - [group_id:7052, {kafka/rxdata-thirdpartybeacon/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 47] raft - [group_id:6793, {kafka/rx-confiant-response/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 31] raft - [group_id:7307, {kafka/geouniques/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 34] raft - [group_id:6750, {kafka/rx-cm-tremor/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 45] raft - [group_id:6999, {kafka/rxdata-rg_application/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 40] raft - [group_id:6772, {kafka/rx-confiant-request-iad/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 41] raft - [group_id:6774, {kafka/rx-confiant-request-iad/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 54] raft - [group_id:7027, {kafka/rxdata-segmentusage/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 40] raft - [group_id:6982, {kafka/rxdata-pubopportunity/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 41] raft - [group_id:6985, {kafka/rxdata-pubopportunity/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 54] raft - [group_id:6817, {kafka/rx-uid-map-batch/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 9] raft - [group_id:6669, {kafka/pixelverse_conv_diagnostic/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 56] raft - [group_id:7035, {kafka/rxdata-sspbidresponse/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 36] raft - [group_id:7326, {kafka/rx-uid-map-batch/15}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 22] raft - [group_id:6712, {kafka/rx-aerokeys-userdata/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 56] raft - [group_id:6824, {kafka/rx-usersync-us/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 36] raft - [group_id:6968, {kafka/rxdata-outstream_pread_event/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 8] raft - [group_id:7088, {kafka/summarystats-dspstat/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 22] raft - [group_id:6920, {kafka/rxdata-impexpired/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 22] raft - [group_id:7136, {kafka/__consumer_offsets/15}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 8] raft - [group_id:6875, {kafka/rxdata-dspbidrequest/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 6] raft - [group_id:7081, {kafka/summarystats-dspstat/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 2] raft - [group_id:6645, {kafka/dmp-cookie-sync/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 4] raft - [group_id:7073, {kafka/rxdata-videoevent/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 27] raft - [group_id:7296, {kafka/rx-fc-iad2/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 19] raft - [group_id:6702, {kafka/rx-aerokeys-ipdata/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 6] raft - [group_id:6656, {kafka/pixelverse_conv_ap/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 51] raft - [group_id:6808, {kafka/rx-uid-map/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 27] raft - [group_id:6727, {kafka/rx-bidrequest-stream/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 6] raft - [group_id:6867, {kafka/rxdata-click/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 19] raft - [group_id:6912, {kafka/rxdata-impexpired/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 16] raft - [group_id:6900, {kafka/rxdata-impbeacon/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 49] raft - [group_id:7012, {kafka/rxdata-rg_report/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 19] raft - [group_id:7123, {kafka/__consumer_offsets/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 29] raft - [group_id:6736, {kafka/rx-cm-post/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 62] raft - [group_id:6844, {kafka/rxdata-bidresponse/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 18] raft - [group_id:6909, {kafka/rxdata-impconfirmation/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 29] raft - [group_id:7303, {kafka/geouniques/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 5] raft - [group_id:6865, {kafka/rxdata-click/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 30] raft - [group_id:7304, {kafka/geouniques/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 62] raft - [group_id:7055, {kafka/rxdata-thirdpartybeacon/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 12] raft - [group_id:7101, {kafka/summarystats-supplystat/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 63] raft - [group_id:6847, {kafka/rxdata-bidresponse/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 42] raft - [group_id:6779, {kafka/rx-confiant-request-iad/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 35] raft - [group_id:6756, {kafka/rx-confiant-request-ams/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 12] raft - [group_id:6677, {kafka/pixelverse_conv_eu/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 50] raft - [group_id:7013, {kafka/rxdata-rg_report/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 52] raft - [group_id:6813, {kafka/rx-uid-map-batch/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 13] raft - [group_id:6892, {kafka/rxdata-impbeacon/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 57] raft - [group_id:6827, {kafka/rx-usersync-us/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 38] raft - [group_id:6766, {kafka/rx-confiant-request-ap/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 13] raft - [group_id:7105, {kafka/summarystats-supplystat/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 21] raft - [group_id:7130, {kafka/__consumer_offsets/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 31] raft - [group_id:6740, {kafka/rx-cm-post/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 34] raft - [group_id:7320, {kafka/rx-fc-usw/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 45] raft - [group_id:6788, {kafka/rx-confiant-request-usw/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 26] raft - [group_id:6725, {kafka/rx-bidrequest-stream/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 9] raft - [group_id:7090, {kafka/summarystats-dspstat/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 9] raft - [group_id:6877, {kafka/rxdata-dspbidrequest/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 26] raft - [group_id:7290, {kafka/rx-fc-iad/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 36] raft - [group_id:6758, {kafka/rx-confiant-request-ams/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 8] raft - [group_id:6666, {kafka/pixelverse_conv_diagnostic/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 37] raft - [group_id:6972, {kafka/rxdata-providerdmpevent/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 32] raft - [group_id:7313, {kafka/rx-fc-usw/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 2] raft - [group_id:6854, {kafka/rxdata-bidresponse_extra/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 23] raft - [group_id:6923, {kafka/rxdata-impnurl/2}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 37] raft - [group_id:6762, {kafka/rx-confiant-request-ap/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 11] raft - [group_id:7099, {kafka/summarystats-pubdealstat/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 25] raft - [group_id:7289, {kafka/rx-fc-iad/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 37] raft - [group_id:7330, {kafka/rx-uid-map-batch/19}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 44] raft - [group_id:6784, {kafka/rx-confiant-request-usw/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 18] raft - [group_id:7121, {kafka/__consumer_offsets/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 17] raft - [group_id:6694, {kafka/rx-aerokeys-devicedata/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 20] raft - [group_id:7128, {kafka/__consumer_offsets/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 30] raft - [group_id:6738, {kafka/rx-cm-post/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 10] raft - [group_id:6882, {kafka/rxdata-dvviewability/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 63] raft - [group_id:7058, {kafka/rxdata-thirdpartybeacon/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 15] raft - [group_id:6898, {kafka/rxdata-impbeacon/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 38] raft - [group_id:6976, {kafka/rxdata-providerdmpevent/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 21] raft - [group_id:6709, {kafka/rx-aerokeys-ipdata/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 55] raft - [group_id:7031, {kafka/rxdata-sspbidresponse/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 53] raft - [group_id:7024, {kafka/rxdata-segmentusage/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 28] raft - [group_id:7297, {kafka/rx-fc-iad2/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 55] raft - [group_id:6821, {kafka/rx-usersync-us/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 4] raft - [group_id:6650, {kafka/dmp-cookie-sync/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 43] raft - [group_id:6780, {kafka/rx-confiant-request-iad/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 39] raft - [group_id:6767, {kafka/rx-confiant-request-ap/6}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 27] raft - [group_id:6938, {kafka/rxdata-impserve/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 23] raft - [group_id:7281, {kafka/rx-fc-iad/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 49] raft - [group_id:6802, {kafka/rx-uid-map/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 3] raft - [group_id:6858, {kafka/rxdata-bidresponse_extra/7}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 5] raft - [group_id:7079, {kafka/rxdata-videoevent/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 3] raft - [group_id:7072, {kafka/rxdata-videoevent/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 20] raft - [group_id:6705, {kafka/rx-aerokeys-ipdata/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 30] raft - [group_id:6949, {kafka/rxdata-lurl/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 10] raft - [group_id:6672, {kafka/pixelverse_conv_eu/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 25] raft - [group_id:6720, {kafka/rx-aerokeys-userdata/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 16] raft - [group_id:6691, {kafka/rx-aerokeys-devicedata/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 7] raft - [group_id:6661, {kafka/pixelverse_conv_diagnostic/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,782 [shard 14] raft - [group_id:7109, {kafka/summarystats-supplystat/8}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,783 [shard 14] raft - [group_id:6895, {kafka/rxdata-impbeacon/4}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,783 [shard 29] raft - [group_id:6946, {kafka/rxdata-lurl/5}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,783 [shard 35] raft - [group_id:7321, {kafka/rx-uid-map-batch/10}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,781 [shard 1] raft - [group_id:7064, {kafka/rxdata-usersync/3}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,788 [shard 1] raft - [group_id:6852, {kafka/rxdata-bidresponse_extra/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,788 [shard 1] raft - [group_id:6642, {kafka/dmp-cookie-sync/1}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,796 [shard 58] raft - [group_id:7041, {kafka/rxdata-testad/0}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,796 [shard 58] raft - [group_id:6830, {kafka/rx-usersync-us/9}] consensus.cc:239 - Stopping
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: Reactor stalled for 97 ms on shard 61. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbba2 0x194edf7 0x43a477a 0x43f031a 0x19ac73f 0x345f55c 0x194254a 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa13d02ff 0xffffffffa13d0511 0xffffffffc02c7b9d 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: Reactor stalled for 97 ms on shard 13. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbba2 0x194edf7 0x43a477a 0x43f031a 0x19ac73f 0x345f55c 0x194254a 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack: 0xffffffffffffff80 0xffffffffa150092e 0xffffffffc02e7f40 0xffffffffc02ef919 0xffffffffc02e80fb 0xffffffffc02e8630 0xffffffffc02d57d5 0xffffffffa136aa1e 0xffffffffa136ae45 0xffffffffa136b417 0xffffffffc02c7331 0xffffffffc02c7b6b 0xffffffffc02c7cf2 0xffffffffa13b13c6 0xffffffffa13b20ca 0xffffffffa13b5338 0xffffffffa13b5b67 0xffffffffa1ab314b 0xffffffffa1c0009b
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: Reactor stalled for 155 ms on shard 61. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbba2 0x194edf7 0x43a477a 0x43f031a 0x19ac73f 0x345f55c 0x194254a 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: kernel callstack:
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 0] cluster - leader_balancer.cc:195 - Stopping Leader Balancer...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 0] cluster - partition_balancer_backend.cc:101 - stopping...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 0] cluster - metrics_reporter.cc:130 - Stopping Metrics Reporter...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 0] cluster - feature_manager.cc:159 - Stopping Feature Manager...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 0] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 3] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 19] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 56] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 37] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 5] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 54] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 32] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 18] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 47] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 26] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 27] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 59] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 43] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 29] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 44] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 40] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 7] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 57] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 11] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 15] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 23] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 38] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 35] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 24] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 30] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 28] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 36] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 6] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 21] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 45] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 9] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 16] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 50] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 51] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 52] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 8] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 2] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 33] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 63] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 60] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 25] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 39] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 58] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 13] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 4] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 42] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 41] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 46] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 62] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 20] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 34] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:21 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 10] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 49] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 61] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 48] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 31] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 1] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 14] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 12] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 53] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 55] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:21,994 [shard 22] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,000 [shard 17] cluster - health_monitor_frontend.cc:33 - Stopping Health Monitor Frontend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 0] cluster - health_monitor_backend.cc:118 - Stopping Health Monitor Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 0] cluster - health_manager.cc:57 - Stopping Health Manager...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 0] cluster - members_backend.cc:64 - Stopping Members Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 0] cluster - config_manager.cc:220 - Stopping Config Manager...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 0] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 2] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 5] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 6] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 4] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 15] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 21] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 22] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 10] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 44] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 53] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 31] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 23] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 8] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 20] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 29] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 47] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 33] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 36] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 51] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 56] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 24] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 61] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 14] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 11] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 16] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 37] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 38] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 46] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 49] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 7] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 41] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 54] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 25] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 57] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 19] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 39] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 26] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 63] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 32] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 30] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 18] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 45] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 1] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 13] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 55] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 42] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 52] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 34] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 40] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 3] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 59] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 50] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 48] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 12] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 60] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 43] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 62] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 27] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 28] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 58] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 35] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,002 [shard 9] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,008 [shard 17] cluster - controller_backend.cc:268 - Stopping Controller Backend...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 0] cluster - members_manager.cc:344 - stopping cluster::members_manager...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 0] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 3] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 6] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 12] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 13] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 15] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 16] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 7] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 24] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 23] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 31] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 10] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 4] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 34] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 38] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 43] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 47] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 52] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 57] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 17] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 62] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 63] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 25] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 19] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 60] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 21] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 22] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 1] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 28] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 30] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 27] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 29] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 32] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 33] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 42] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 11] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 35] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 36] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 2] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 37] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 14] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 39] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 40] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 41] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 5] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 48] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 50] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 51] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 45] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 49] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 46] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 55] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 53] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 8] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 54] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 44] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 56] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 58] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 59] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 61] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 26] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 20] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,014 [shard 9] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,020 [shard 18] cluster - drain_manager.cc:30 - Drain manager stopping
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 0] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 2] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 5] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 9] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 11] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 13] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 14] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 8] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 6] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 7] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 18] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 22] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 26] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 28] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 4] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 12] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 34] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 38] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 45] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 47] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 49] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 48] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 54] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 55] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 63] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 27] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 3] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 1] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 29] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 30] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 31] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 33] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 16] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 15] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 32] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 37] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 50] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 62] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 40] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 19] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 60] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 42] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 44] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 56] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 58] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 53] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 24] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 59] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 25] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 23] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 36] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 43] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 39] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 57] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 35] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 21] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 41] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 46] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 61] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 10] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 20] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 51] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,025 [shard 52] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,031 [shard 17] cluster - partition_leaders_table.cc:30 - Stopping Partition Leaders Table...
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 0] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/0_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 1] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/1_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 13] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/13_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 54] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/54_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 26] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/26_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 6] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/6_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 41] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/41_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 33] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/33_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 50] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/50_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 57] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/57_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 30] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/30_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 37] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/37_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 23] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/23_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 40] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/40_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 59] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/59_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 43] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/43_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 28] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/28_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 46] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/46_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 3] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/3_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 7] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/7_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 45] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/45_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 24] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/24_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 12] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/12_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 5] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/5_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 2] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/2_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 29] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/29_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 19] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/19_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 32] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/32_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 31] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/31_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 36] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/36_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 34] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/34_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 8] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/8_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 39] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/39_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 27] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/27_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 15] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/15_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 49] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/49_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 53] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/53_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 55] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/55_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 56] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/56_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 16] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/16_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 58] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/58_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 18] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/18_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 38] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/38_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 35] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/35_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 52] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/52_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 17] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/17_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 60] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/60_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 47] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/47_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 51] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/51_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 22] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/22_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 61] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/61_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 48] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/48_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 62] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/62_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 63] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/63_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 21] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/21_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 25] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/25_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 10] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/10_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 14] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/14_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 4] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/4_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 11] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/11_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,044 [shard 20] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/20_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 42] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/42_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 9] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/9_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,038 [shard 44] kvstore - kvstore.cc:106 - Stopping kvstore: dir /var/lib/redpanda/data/redpanda/kvstore/44_0
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io rpk[5055]: INFO  2022-11-17 08:36:22,063 [shard 0] redpanda::main - application.cc:239 - Shutdown complete.
Nov 17 08:36:22 srv-01-03-404.iad1.trmr.io systemd[1]: Stopped Redpanda, the fastest queue in the West..

[Thu Nov 17 14:03:07 2022] reactor-27[63448]: segfault at 18 ip 0000564710ce56da sp 00007f7867df1b80 error 4 in redpanda[56470da45000+353c000] [Thu Nov 17 14:03:07 2022] Code: b4 de f8 fb ff ff 48 81 c3 7f ff ff ff 49 89 9f a0 04 00 00 48 89 e3 4c 39 f3 74 24 0f 1f 40 00 4c 8b 2b 49 8b 45 00 4c 89 ef <ff> 50 18 49 8b 45 00 4c 89 ef ff 50 08 48 83 c3 08 4c 39 f3 75 e0 [Thu Nov 17 14:05:07 2022] reactor-60[13816]: segfault at 1b1de625 ip 000056422d4612e3 sp 00007f4d2f5ef8c0 error 4 in redpanda[56422a2d8000+353c000] [Thu Nov 17 14:05:07 2022] Code: c1 e1 04 44 39 bc 0f 88 01 00 00 0f 82 77 11 00 00 4c 8d a4 0f 90 01 00 00 48 8b 9c 0f 90 01 00 00 48 85 db 0f 84 ed 04 00 00 <48> 8b 03 49 89 04 24 48 ff 8c 0f 98 01 00 00 e9 b0 0b 00 00 48 8b [Thu Nov 17 14:05:49 2022] reactor-60[14270]: segfault at 1be5cc ip 0000559dd668b90c sp 00007f0385bf0f50 error 4 in redpanda[559dd33e2000+353c000] [Thu Nov 17 14:05:49 2022] Code: ca 8d 00 00 83 f8 ff 74 46 89 c5 85 c0 74 38 89 eb 4c 8d 64 24 30 0f 1f 84 00 00 00 00 00 49 8b 7c 24 f0 49 8b 34 24 48 8b 07 <ff> 10 49 83 c4 20 48 ff cb 75 e9 0f 57 c0 0f 29 44 24 10 b3 01 81 [Thu Nov 17 14:06:04 2022] reactor-58[14775]: segfault at 4 ip 000055bc5b6142e3 sp 00007fd7fc9ef660 error 4 in redpanda[55bc5848b000+353c000] [Thu Nov 17 14:06:04 2022] Code: c1 e1 04 44 39 bc 0f 88 01 00 00 0f 82 77 11 00 00 4c 8d a4 0f 90 01 00 00 48 8b 9c 0f 90 01 00 00 48 85 db 0f 84 ed 04 00 00 <48> 8b 03 49 89 04 24 48 ff 8c 0f 98 01 00 00 e9 b0 0b 00 00 48 8b [Thu Nov 17 14:08:27 2022] traps: reactor-59[15564] general protection fault ip:561e0c0a243f sp:7fb0b69eff10 error:0 in redpanda[561e0b2a2000+353c000] [Thu Nov 17 14:09:11 2022] traps: reactor-60[16370] general protection fault ip:5593696c42e3 sp:7f589b3ee300 error:0 in redpanda[55936653b000+353c000] [Thu Nov 17 14:09:51 2022] traps: reactor-60[16841] general protection fault ip:563d99e25893 sp:7f80b2deead0 error:0 in redpanda[563d96c9c000+353c000] [Thu Nov 17 14:11:27 2022] reactor-59[17285]: segfault at 4f2 ip 00005558986d82e3 sp 00007fece99ee380 error 4 in redpanda[55589554f000+353c000] [Thu Nov 17 14:11:27 2022] Code: c1 e1 04 44 39 bc 0f 88 01 00 00 0f 82 77 11 00 00 4c 8d a4 0f 90 01 00 00 48 8b 9c 0f 90 01 00 00 48 85 db 0f 84 ed 04 00 00 <48> 8b 03 49 89 04 24 48 ff 8c 0f 98 01 00 00 e9 b0 0b 00 00 48 8b [Thu Nov 17 14:12:08 2022] reactor-60[18213]: segfault at 1c90ea ip 0000561eec4d62e3 sp 00007fd6855ef8c0 error 4 in redpanda[561ee934d000+353c000] [Thu Nov 17 14:12:08 2022] Code: c1 e1 04 44 39 bc 0f 88 01 00 00 0f 82 77 11 00 00 4c 8d a4 0f 90 01 00 00 48 8b 9c 0f 90 01 00 00 48 85 db 0f 84 ed 04 00 00 <48> 8b 03 49 89 04 24 48 ff 8c 0f 98 01 00 00 e9 b0 0b 00 00 48 8b [Thu Nov 17 14:13:41 2022] traps: reactor-59[18645] general protection fault ip:5589e183837b sp:7f55ffbef4b0 error:0 in redpanda[5589de6f5000+353c000] [Thu Nov 17 14:15:52 2022] traps: reactor-60[20232] general protection fault ip:555b71ab22e3 sp:7fa760dee790 error:0 in redpanda[555b6e929000+353c000] [Thu Nov 17 14:16:29 2022] traps: reactor-60[20705] general protection fault ip:55d8db5f709a sp:7f556ebef840 error:0 in redpanda[55d8da7f3000+353c000] [Thu Nov 17 14:18:03 2022] traps: reactor-59[21164] general protection fault ip:561a8084f2e3 sp:7fa15b1edf40 error:0 in redpanda[561a7d6c6000+353c000] [Thu Nov 17 14:19:35 2022] reactor-59[21921]: segfault at 2c7 ip 00005587ed2dfdf8 sp 00007f5ee31efd50 error 4 in redpanda[5587ecd3f000+353c000] [Thu Nov 17 14:19:35 2022] Code: 10 48 39 ef 74 10 48 8b 75 00 48 85 f6 74 0c 48 8b 5c 08 68 eb 14 48 89 c6 eb 63 48 89 6d 00 48 89 6c 08 68 48 89 eb 48 89 ee <4c> 8b 4e 08 48 89 7e 08 4c 89 4c 24 10 48 8b 33 48 89 3b 48 8b 5d [Thu Nov 17 14:20:52 2022] reactor-60[23170]: segfault at 1c5a0e ip 000055e7ff500893 sp 00007ff3acbee790 error 4 in redpanda[55e7fc377000+353c000] [Thu Nov 17 14:20:52 2022] Code: 0f 44 c9 c7 01 00 00 00 00 41 89 30 48 8b 4c 07 18 48 85 c9 74 b1 4c 8d 54 07 18 0f b7 7c 07 02 0f 1f 80 00 00 00 00 48 89 cb <48> 8b 09 48 89 2b 48 ff c2 ff c7 48 89 dd 48 85 c9 75 ea 49 c7 02

Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0-63 Thread(s) per core: 1 Core(s) per socket: 64 Socket(s): 1 NUMA node(s): 1 Vendor ID: AuthenticAMD CPU family: 25 Model: 1 Model name: AMD EPYC 7763 64-Core Processor Stepping: 1 CPU MHz: 3240.760 BogoMIPS: 4890.28 Virtualization: AMD-V L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 32768K NUMA node0 CPU(s): 0-63 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin brs arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca

Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: Segmentation fault on shard 58. Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: Backtrace: Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4ab1196 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4b13ff6 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: /opt/redpanda/lib/libc.so.6+0x42abf Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4a212e2 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4a2f025 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x49be4ce Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x44f8c65 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x44fa423 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x442482b Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4419ce4 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x43ff4d0 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4404aa3 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x43ffa94 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x441b55e Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x441dfbc Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4625ccc Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x3441e09 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x344163a Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4ace59f Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4ad2277 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4b15745 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: 0x4a6f10f Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: /opt/redpanda/lib/libc.so.6+0x91016 Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io rpk[62143]: /opt/redpanda/lib/libc.so.6+0x1166cf Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io systemd[1]: redpanda.service: main process exited, code=killed, status=11/SEGV Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io systemd[1]: Unit redpanda.service entered failed state. Nov 17 14:29:34 srv-01-09-310.iad1.trmr.io systemd[1]: redpanda.service failed. Nov 17 14:29:35 srv-01-09-310.iad1.trmr.io systemd[1]: redpanda.service holdoff time over, scheduling restart. Nov 17 14:29:35 srv-01-09-310.iad1.trmr.io systemd[1]: Stopped Redpanda, the fastest queue in the West..

SeqTest: (g=0): rw=rw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=psync, iodepth=1 fio-3.7 Starting 1 process Jobs: 1 (f=1): [M(1)][100.0%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 00m:00s] SeqTest: (groupid=0, jobs=1): err= 0: pid=28734: Thu Nov 17 14:55:03 2022 read: IOPS=159k, BW=2486MiB/s (2607MB/s)(1460GiB/601342msec) clat (nsec): min=641, max=4805.0k, avg=1291.16, stdev=3192.57 lat (nsec): min=671, max=4806.0k, avg=1317.03, stdev=3192.75 clat percentiles (nsec): | 1.00th=[ 956], 5.00th=[ 1048], 10.00th=[ 1064], 20.00th=[ 1096], | 30.00th=[ 1160], 40.00th=[ 1176], 50.00th=[ 1192], 60.00th=[ 1208], | 70.00th=[ 1256], 80.00th=[ 1368], 90.00th=[ 1560], 95.00th=[ 2040], | 99.00th=[ 2352], 99.50th=[ 2512], 99.90th=[ 3312], 99.95th=[ 3728], | 99.99th=[39168] bw ( MiB/s): min= 17, max= 5193, per=100.00%, avg=2986.51, stdev=1055.98, samples=1001 iops : min= 1144, max=332368, avg=191136.34, stdev=67583.01, samples=1001 write: IOPS=159k, BW=2486MiB/s (2607MB/s)(1460GiB/601342msec) clat (nsec): min=1192, max=501396, avg=2829.37, stdev=1108.78 lat (nsec): min=1252, max=501466, avg=2897.68, stdev=1110.21 clat percentiles (nsec): | 1.00th=[ 1576], 5.00th=[ 1640], 10.00th=[ 1640], 20.00th=[ 1688], | 30.00th=[ 2800], 40.00th=[ 2864], 50.00th=[ 2896], 60.00th=[ 2960], | 70.00th=[ 3152], 80.00th=[ 3440], 90.00th=[ 3664], 95.00th=[ 4080], | 99.00th=[ 5920], 99.50th=[ 6688], 99.90th=[ 7904], 99.95th=[ 8512], | 99.99th=[11072] bw ( MiB/s): min= 17, max= 5181, per=100.00%, avg=2986.78, stdev=1056.37, samples=1001 iops : min= 1148, max=331616, avg=191153.66, stdev=67607.49, samples=1001 lat (nsec) : 750=0.07%, 1000=0.65% lat (usec) : 2=60.36%, 4=36.17%, 10=2.74%, 20=0.01%, 50=0.01% lat (usec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01% lat (msec) : 2=0.01%, 4=0.01%, 10=0.01% cpu : usr=10.57%, sys=89.14%, ctx=688789, majf=0, minf=13 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=95673882,95682561,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=2486MiB/s (2607MB/s), 2486MiB/s-2486MiB/s (2607MB/s-2607MB/s), io=1460GiB (1568GB), run=601342-601342msec WRITE: bw=2486MiB/s (2607MB/s), 2486MiB/s-2486MiB/s (2607MB/s-2607MB/s), io=1460GiB (1568GB), run=601342-601342msec Disk stats (read/write): md0: ios=169001/4620401, merge=0/0, ticks=68270/766698858, in_queue=766767128, util=25.78%, aggrios=21125/1156355, aggrmerge=0/0, aggrticks=8399/11310365, aggrin_queue=11318764, aggrutil=25.01% nvme3n1: ios=18537/1155697, merge=0/0, ticks=7023/11307406, in_queue=11314429, util=22.22% nvme7n1: ios=18059/1160316, merge=0/0, ticks=7161/11554948, in_queue=11562108, util=22.31% nvme0n1: ios=25626/1153543, merge=0/0, ticks=9329/11974729, in_queue=11984057, util=22.38% nvme4n1: ios=22202/1155866, merge=0/0, ticks=7565/11307487, in_queue=11315051, util=22.57% nvme1n1: ios=19085/1153543, merge=0/0, ticks=7693/12017526, in_queue=12025219, util=22.42% nvme5n1: ios=21216/1155866, merge=0/0, ticks=10610/8896171, in_queue=8906782, util=22.32% nvme2n1: ios=24798/1155697, merge=0/0, ticks=10096/11630921, in_queue=11641018, util=25.01% nvme6n1: ios=19478/1160316, merge=0/0, ticks=7718/11793732, in_queue=11801450, util=22.53%

Nov 18 11:21:08 srv-01-03-404.iad1.trmr.io rpk[9395]: ERROR 2022-11-18 11:21:08,227 [shard 0] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-xfs-builders-i-06c4550a08f45c0ae-1/redpanda/redpanda/src/v/cluster/scheduling/allocation_node.cc:81) 'domain_partitions > allocation_capacity{0} && domain_partitions <= _allocated_partitions' Unable to deallocate partition from core 27 in domain 0 at node {node: 8, max_partitions_per_core: 7000, state: active, partition_capacity: 447993, weights: [(2)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(1)(1)(1)(1)(1)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)(0)], allocated: 5({0: 0, -1: 5})}

Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: -------- Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,246 [shard 14] r/heartbeat - heartbeat_manager.cc:209 - Closed unresponsive connection to 1 Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: WARN 2022-11-18 22:37:27,249 [shard 14] raft - [follower: {id: {1}, revision: {1248}}] [group_id:6895, {kafka/rxdata-impbeacon/4}] - recovery_stm.cc:435 - recovery append entries error: rpc::errc::disconnected_endpoint(node down) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,249 [shard 61] r/heartbeat - heartbeat_manager.cc:209 - Closed unresponsive connection to 1 Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,249 [shard 14] raft - [group_id:6895, {kafka/rxdata-impbeacon/4}] consensus.cc:539 - Node {id: {1}, revision: {1248}} recovery cancelled (rpc::errc::disconnected_endpoint(node down)) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,258 [shard 26] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impserve/2_1256/2872337596-33-v1.log Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,258 [shard 9] r/heartbeat - heartbeat_manager.cc:209 - Closed unresponsive connection to 1 Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,276 [shard 17] r/heartbeat - heartbeat_manager.cc:209 - Closed unresponsive connection to 1 Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,289 [shard 55] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/0_1234/8629374553-55-v1.log Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,321 [shard 59] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-bidrequest/3_1236/523306929-31-v1.log Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,351 [shard 13] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,351 [shard 55] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,351 [shard 7] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,352 [shard 29] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,354 [shard 58] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/9_1234/8637243084-44-v1.log Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,362 [shard 9] rpc - server.cc:135 - vectorized internal rpc protocol - Disconnected 10.16.67.49:50057 (applying protocol, Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,362 [shard 28] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,368 [shard 13] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: INFO 2022-11-18 22:37:27,368 [shard 7] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:37:27 srv-01-03-404.iad1.trmr.io rpk[59281]: WARN 2022-11-18 22:37:27,370 [shard 59] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf

Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: Reactor stalled for 94 ms on shard 60. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450bde9 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa84f276e 0xffffffffa81607e9 0xffffffffa807fbbb 0xffffffffa807fd5d 0xffffffffa82c152e 0xffffffffa82a819d 0xffffffffa82abdd7 0xffffffffa82b0703 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa84f276e 0xffffffffa81607e9 0xffffffffa807fbbb 0xffffffffa807fd5d 0xffffffffa82c152e 0xffffffffa82a819d 0xffffffffa82abdd7 0xffffffffa82b0703 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa82d30ad 0xffffffffa82d6f07 0xffffffffa82d799b 0xffffffffa82f9606 0xffffffffa8318862 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa84b1685 0xffffffffa84a247e 0xffffffffa8492a2c 0xffffffffa8492f15 0xffffffffa84932ef 0xffffffffa849363a 0xffffffffa83cf672 0xffffffffa83cfac2 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa83ac8c6 0xffffffffa83482a0 0xffffffffa8348607 0xffffffffa83486aa 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffc057beb0 0xffffffffc057d4b2 0xffffffffa84a1e79 0xffffffffa8493b20 0xffffffffa8493d9a 0xffffffffa83d0295 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa82a379c 0xffffffffa82a3c8c 0xffffffffa82a580e 0xffffffffa82a5a84 0xffffffffa84ed656 0xffffffffa848e41d 0xffffffffa83cfa56 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa84b53d0 0xffffffffa84b567e 0xffffffffa848c082 0xffffffffa848db81 0xffffffffa83cf729 0xffffffffa83cfa19 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa82a379c 0xffffffffa82a3c8c 0xffffffffa82a580e 0xffffffffa82a5a84 0xffffffffa84ed656 0xffffffffa848e41d 0xffffffffa83cfa56 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa8493279 0xffffffffc0a093f2 0xffffffffc0a0b16d 0xffffffffa87db0b0 0xffffffffa87db24d 0xffffffffa84929fb 0xffffffffa8492f15 0xffffffffa84932ef 0xffffffffa849363a 0xffffffffa83cf672 0xffffffffa83cfac2 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854ca93 0xffffffffa83b5b26 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,609 [shard 44] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-confiant-request-usw/4_1226/18821239-64-v1.log Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,610 [shard 27] raft - [group_id:7295, {kafka/rx-fc-iad2/4}] vote_stm.cc:279 - became the leader term: 56 Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,612 [shard 24] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-aerokeys-userdata/8_1212/55-58-v1.log Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,631 [shard 60] raft - [group_id:6838, {kafka/rxdata-bidrequest/7}] vote_stm.cc:77 - vote reply from {id: {7}, revision: {1236}} - {term:{32}, target_node_id{id: {5}, revision: {1236}}, vote_granted: 1, log_ok:1} Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: WARN 2022-11-18 22:40:06,631 [shard 60] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: -------- Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,632 [shard 47] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-confiant-response/3_1228/1703499-47-v1.log Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,648 [shard 50] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-uid-map/4_1230/22226950-71-v1.log Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,665 [shard 28] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-cm-post/0_1216/44-51-v1.log Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,729 [shard 10] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-dspbidrequest/9_1244/1221661334-47-v1.log Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: WARN 2022-11-18 22:40:06,868 [shard 10] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: -------- Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 18 22:40:06 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:06,903 [shard 60] raft - [group_id:6838, {kafka/rxdata-bidrequest/7}] vote_stm.cc:279 - became the leader term: 32 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:07,084 [shard 27] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-bidrequest-stream/7_1214/21190070168-46-v1.log Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: Reactor stalled for 110 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450a89f 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa84f2763 0xffffffffa81607e9 0xffffffffa807fbbb 0xffffffffa807fd5d 0xffffffffa82c152e 0xffffffffa82a819d 0xffffffffa82abdd7 0xffffffffa82b0703 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa83caa69 0xffffffffa83cff06 0xffffffffa83d0511 0xffffffffc075ab9d 0xffffffffc075acf2 0xffffffffa83b13c6 0xffffffffa83b20ca 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa848c03d 0xffffffffa848db81 0xffffffffa83cf729 0xffffffffa83cfa19 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1134 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1170 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa832eba6 0xffffffffa8277a06 0xffffffffa8277b6d 0xffffffffa8272168 0xffffffffa8276e6f 0xffffffffa83164e6 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa82697b9 0xffffffffc0a0b0e6 0xffffffffa87db0b0 0xffffffffa87db24d 0xffffffffa84929fb 0xffffffffa8492f15 0xffffffffa84932ef 0xffffffffa849363a 0xffffffffa83cf672 0xffffffffa83cfac2 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa88365b5 0xffffffffa8836617 0xffffffffa8836687 0xffffffffa8836715 0xffffffffa88367ec 0xffffffffa883685f 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa84ece15 0xffffffffa848e5bc 0xffffffffa83cfa56 0xffffffffa83d022e 0xffffffffa83d0511 0xffffffffc0759f6a 0xffffffffc075a135 0xffffffffa83b177f 0xffffffffa83b2027 0xffffffffa83b5338 0xffffffffa83b5b67 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854bfe8 0xffffffffa8921b87 0xffffffffa8921e8d 0xffffffffa895e303 0xffffffffa8833fc2 0xffffffffa883431c 0xffffffffa8836731 0xffffffffa88367ec 0xffffffffa883685f 0xffffffffa8ab314b 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa81369c6 0xffffffffa8ab3158 0xffffffffa8c0009b Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: 0xffffffffffffff80 0xffffffffa854b907 0xffffffffa82b1134 0xffffffffa83163fa 0xffffffffa8318879 0xffffffffa82b0725 0xffffffffa82b080b 0xffffffffa807c7fe 0xffffffffa8ab729c 0xffffffffa8c00b77 Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: Reactor stalled for 167 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbe42 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: Reactor stalled for 275 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450bf1d 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:07 srv-01-08-309.iad1.trmr.io rpk[54634]: kernel callstack: Nov 18 22:40:08 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:08,421 [shard 57] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/6_1234/8827742883-53-v1.log Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,171 [shard 21] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,171 [shard 59] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,173 [shard 18] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,173 [shard 48] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,173 [shard 19] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,173 [shard 55] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,203 [shard 19] rpc - Disconnected from server {host: 10.16.67.49, port: 33145}: std::__1::system_error (error system:104, read: Connection reset by peer) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,552 [shard 7] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:20, base_offset:1137681689, committed_offset:1137795840, dirty_offset:1137795840}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rxdata-dspbidrequest/0_1244/1137681689-20-v1.log, (105487674 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rxdata-dspbidrequest/0_1244/1137681689-20-v1.base_index, offsets:{1137681689}, index:{header_bitflags:0, base_offset:{1137681689}, max_offset:{1137795840}, base_timestamp:{timestamp: 1668569666303}, max_timestamp:{timestamp: 1668570007198}, index(3025,3025,3025)}, step:32768, needs_persistence:0}} Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,552 [shard 7] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rxdata-dspbidrequest/0_1244/1137681689-20-v1.log", "/var/lib/redpanda/data/kafka/rxdata-dspbidrequest/0_1244/1137681689-20-v1.base_index"} Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,583 [shard 7] offset_translator - ntp: {kafka/rxdata-dspbidrequest/0} - offset_translator.cc:277 - prefix_truncate at offset: 1137795840, new state: {base offset/delta: {1137795840}/19, map size: 25, last delta: 43} Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,598 [shard 61] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.141:23407 (shutting down, Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,598 [shard 9] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.76:38672 (shutting down, Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,599 [shard 48] kafka - connection_context.cc:371 - Disconnected 10.16.67.58:10392 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,599 [shard 41] kafka - connection_context.cc:371 - Disconnected 10.16.65.51:59380 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,599 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.67.72:33098 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,599 [shard 48] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.58:10392 (shutting down, Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,600 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.66.158:49288 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,600 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.66.158:49288 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,602 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.66.158:49288 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,602 [shard 33] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.22:25120 (shutting down, Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,602 [shard 27] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.158:49288 (shutting down, Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,605 [shard 9] kafka - connection_context.cc:371 - Disconnected 10.16.66.76:38672 (stream closed) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,605 [shard 33] kafka - connection_context.cc:371 - Disconnected 10.16.67.22:25120 (stream closed) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,605 [shard 61] kafka - connection_context.cc:371 - Disconnected 10.16.66.141:23407 (stream closed) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,605 [shard 48] kafka - connection_context.cc:371 - Disconnected 10.16.66.204:37868 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,605 [shard 48] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.204:37868 (shutting down, Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,606 [shard 15] kafka - connection_context.cc:371 - Disconnected 10.16.67.70:47286 (Broken pipe) Nov 18 22:40:09 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:09,607 [shard 33] kafka - connection_context.cc:371 - Disconnected 10.16.67.22:25120 (stream closed) Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,655 [shard 58] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-usersync-us/9_1234/8823716823-57-v1.log Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,750 [shard 13] raft - [group_id:6891, {kafka/rxdata-impbeacon/0}] consensus.cc:1566 - Received vote request with larger term from node {id: {7}, revision: {1248}}, received 45, current 44 Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,850 [shard 13] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impbeacon/0_1248/780914858-45-v1.log Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,934 [shard 20] raft - [group_id:7129, {kafka/__consumer_offsets/8}] vote_stm.cc:77 - vote reply from {id: {9}, revision: {1321}} - {term:{45}, target_node_id{id: {5}, revision: {1321}}, vote_granted: 1, log_ok:1} Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,934 [shard 20] raft - [group_id:7129, {kafka/__consumer_offsets/8}] vote_stm.cc:263 - becoming the leader term:45 Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,936 [shard 20] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/__consumer_offsets/8_1321/40-45-v1.log Nov 18 22:40:10 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:10,978 [shard 20] raft - [group_id:7129, {kafka/__consumer_offsets/8}] vote_stm.cc:279 - became the leader term: 45 Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,229 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] vote_stm.cc:77 - vote reply from {id: {0}, revision: {1204}} - {term:{57}, target_node_id{id: {5}, revision: {1204}}, vote_granted: 1, log_ok:1} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,229 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] vote_stm.cc:263 - becoming the leader term:57 Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,230 [shard 11] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_eu/3_1204/54-57-v1.log Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,266 [shard 21] raft - [group_id:6917, {kafka/rxdata-impexpired/6}] consensus.cc:1566 - Received vote request with larger term from node {id: {8}, revision: {1252}}, received 72, current 71 Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,297 [shard 11] raft - [group_id:6674, {kafka/pixelverse_conv_eu/3}] vote_stm.cc:279 - became the leader term: 57 Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,347 [shard 21] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impexpired/6_1252/7373691-72-v1.log Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,352 [shard 26] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:16, base_offset:19744756210, committed_offset:19745468370, dirty_offset:19745468370}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-bidrequest-stream/3_1214/19744756210-16-v1.log, (105801450 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-bidrequest-stream/3_1214/19744756210-16-v1.base_index, offsets:{19744756210}, index:{header_bitflags:0, base_offset:{19744756210}, max_offset:{19745468370}, base_timestamp:{timestamp: 1668569875000}, max_timestamp:{timestamp: 1668570006000}, index(3190,3190,3190)}, step:32768, needs_persistence:0}} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,352 [shard 26] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-bidrequest-stream/3_1214/19744756210-16-v1.log", "/var/lib/redpanda/data/kafka/rx-bidrequest-stream/3_1214/19744756210-16-v1.base_index"} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,395 [shard 26] offset_translator - ntp: {kafka/rx-bidrequest-stream/3} - offset_translator.cc:277 - prefix_truncate at offset: 19745468370, new state: {base offset/delta: {19745468370}/16, map size: 36, last delta: 51} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,632 [shard 29] raft - [group_id:6733, {kafka/rx-cm-post/2}] consensus.cc:1566 - Received vote request with larger term from node {id: {2}, revision: {1216}}, received 57, current 56 Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,750 [shard 25] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:15, base_offset:19745089483, committed_offset:19745789148, dirty_offset:19745789148}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rx-bidrequest-stream/0_1214/19745089483-15-v1.log, (103913983 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rx-bidrequest-stream/0_1214/19745089483-15-v1.base_index, offsets:{19745089483}, index:{header_bitflags:0, base_offset:{19745089483}, max_offset:{19745789148}, base_timestamp:{timestamp: 1668569877000}, max_timestamp:{timestamp: 1668570005000}, index(3134,3134,3134)}, step:32768, needs_persistence:0}} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,750 [shard 25] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rx-bidrequest-stream/0_1214/19745089483-15-v1.log", "/var/lib/redpanda/data/kafka/rx-bidrequest-stream/0_1214/19745089483-15-v1.base_index"} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,790 [shard 25] offset_translator - ntp: {kafka/rx-bidrequest-stream/0} - offset_translator.cc:277 - prefix_truncate at offset: 19745789148, new state: {base offset/delta: {19745789148}/15, map size: 41, last delta: 55} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,797 [shard 29] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rx-cm-post/2_1216/53-57-v1.log Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,893 [shard 5] raft - [group_id:6654, {kafka/pixelverse_conv_ap/3}] vote_stm.cc:77 - vote reply from {id: {8}, revision: {1200}} - {term:{67}, target_node_id{id: {5}, revision: {1200}}, vote_granted: 1, log_ok:1} Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,893 [shard 5] raft - [group_id:6654, {kafka/pixelverse_conv_ap/3}] vote_stm.cc:263 - becoming the leader term:67 Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,894 [shard 5] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/pixelverse_conv_ap/3_1200/63-67-v1.log Nov 18 22:40:11 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:11,979 [shard 5] raft - [group_id:6654, {kafka/pixelverse_conv_ap/3}] vote_stm.cc:279 - became the leader term: 67 Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,276 [shard 9] raft - [group_id:7091, {kafka/summarystats-pubdealstat/0}] vote_stm.cc:77 - vote reply from {id: {4}, revision: {1288}} - {term:{49}, target_node_id{id: {5}, revision: {1288}}, vote_granted: 1, log_ok:1} Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,276 [shard 9] raft - [group_id:7091, {kafka/summarystats-pubdealstat/0}] vote_stm.cc:263 - becoming the leader term:49 Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,277 [shard 9] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/summarystats-pubdealstat/0_1288/8195-49-v1.log Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,302 [shard 12] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.109:45674 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 48] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.8.80:8750 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 37] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.100:60278 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 39] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.115:16046 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 13] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.107:13176 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 11] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.129:41596 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 55] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.0.75:16796 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 39] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.86:42926 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 17] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.65.66:10250 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 46] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.181:62087 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 10] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.0.71:50428 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 24] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.71:22974 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 35] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.129:41458 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,303 [shard 22] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.76:48136 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 61] kafka - connection_context.cc:371 - Disconnected 10.16.67.96:26514 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 42] kafka - connection_context.cc:371 - Disconnected 10.16.67.76:37630 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 23] kafka - connection_context.cc:371 - Disconnected 10.16.67.124:21726 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 43] kafka - connection_context.cc:371 - Disconnected 10.16.66.169:61660 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 22] kafka - connection_context.cc:371 - Disconnected 10.16.66.76:48136 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 13] kafka - connection_context.cc:371 - Disconnected 10.16.67.107:13176 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 1] kafka - connection_context.cc:371 - Disconnected 10.16.67.122:11692 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 6] kafka - connection_context.cc:371 - Disconnected 10.16.66.138:6787 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.16.66.115:16046 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 12] kafka - connection_context.cc:371 - Disconnected 10.16.67.109:45674 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 3] kafka - connection_context.cc:371 - Disconnected 10.25.4.79:23365 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 44] kafka - connection_context.cc:371 - Disconnected 10.16.67.65:7284 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 53] kafka - connection_context.cc:371 - Disconnected 10.16.67.72:21342 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 48] kafka - connection_context.cc:371 - Disconnected 10.25.8.80:8750 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 10] kafka - connection_context.cc:371 - Disconnected 10.16.66.110:13080 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 11] kafka - connection_context.cc:371 - Disconnected 10.16.67.129:41596 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.16.67.117:57240 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.66.110:32538 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.66.181:62087 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 55] kafka - connection_context.cc:371 - Disconnected 10.25.0.75:16796 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 58] kafka - connection_context.cc:371 - Disconnected 10.16.65.65:40482 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 41] kafka - connection_context.cc:371 - Disconnected 10.16.67.30:12106 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 55] kafka - connection_context.cc:371 - Disconnected 10.16.66.91:56196 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 55] kafka - connection_context.cc:371 - Disconnected 10.16.67.118:28824 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.16.67.86:42926 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 37] kafka - connection_context.cc:371 - Disconnected 10.16.67.100:60278 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 22] kafka - connection_context.cc:371 - Disconnected 10.16.67.111:31712 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 17] kafka - connection_context.cc:371 - Disconnected 10.16.65.66:10250 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 37] kafka - connection_context.cc:371 - Disconnected 10.16.66.141:61882 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 22] kafka - connection_context.cc:371 - Disconnected 10.25.4.63:37134 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 63] kafka - connection_context.cc:371 - Disconnected 10.16.66.183:19732 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 38] kafka - connection_context.cc:371 - Disconnected 10.16.67.33:46840 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,304 [shard 35] kafka - connection_context.cc:371 - Disconnected 10.16.67.129:41458 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 58] kafka - connection_context.cc:371 - Disconnected 10.16.67.30:12116 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 47] kafka - connection_context.cc:371 - Disconnected 10.16.67.99:16732 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 10] kafka - connection_context.cc:371 - Disconnected 10.16.67.96:58984 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 15] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.70:47286 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 16] kafka - connection_context.cc:371 - Disconnected 10.16.65.46:32446 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 10] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.96:58984 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 60] kafka - connection_context.cc:371 - Disconnected 10.25.0.62:46110 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 15] kafka - connection_context.cc:371 - Disconnected 10.16.67.70:47286 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 7] kafka - connection_context.cc:371 - Disconnected 10.16.66.141:24145 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 4] kafka - connection_context.cc:371 - Disconnected 10.25.0.39:48946 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 60] kafka - connection_context.cc:371 - Disconnected 10.25.8.78:57386 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 21] kafka - connection_context.cc:371 - Disconnected 10.16.67.96:26468 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 3] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.65.53:37422 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 10] kafka - connection_context.cc:371 - Disconnected 10.25.0.71:50428 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 24] kafka - connection_context.cc:371 - Disconnected 10.16.67.71:22974 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 21] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.96:26468 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 23] kafka - connection_context.cc:371 - Disconnected 10.16.65.71:60228 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.67.72:33098 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 30] kafka - connection_context.cc:371 - Disconnected 10.16.65.67:58692 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 24] kafka - connection_context.cc:371 - Disconnected 10.16.67.33:46938 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 56] kafka - connection_context.cc:371 - Disconnected 10.16.67.118:28438 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,305 [shard 58] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.30:12116 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 7] kafka - connection_context.cc:371 - Disconnected 10.16.67.32:61718 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 7] kafka - connection_context.cc:371 - Disconnected 10.16.66.76:41826 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 41] kafka - connection_context.cc:371 - Disconnected 10.16.67.30:12106 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 47] kafka - connection_context.cc:371 - Disconnected 10.16.67.99:16732 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 41] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.30:12106 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 47] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.99:16732 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.67.19:39612 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.16.67.117:57240 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 24] kafka - connection_context.cc:371 - Disconnected 10.16.67.33:46938 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 24] kafka - connection_context.cc:371 - Disconnected 10.16.67.27:28630 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.67.19:39430 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.66.159:61565 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 33] kafka - connection_context.cc:371 - Disconnected 10.16.66.104:28514 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 39] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.117:57240 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 33] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.104:28514 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 60] kafka - connection_context.cc:371 - Disconnected 10.16.67.85:37684 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 60] kafka - connection_context.cc:371 - Disconnected 10.16.67.118:28560 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 3] kafka - connection_context.cc:371 - Disconnected 10.25.12.68:23496 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 7] kafka - connection_context.cc:371 - Disconnected 10.25.0.71:50464 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 58] kafka - connection_context.cc:371 - Disconnected 10.16.67.30:12116 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,306 [shard 57] kafka - connection_context.cc:371 - Disconnected 10.16.67.124:21558 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,307 [shard 56] kafka - connection_context.cc:371 - Disconnected 10.16.66.191:32808 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,307 [shard 25] kafka - connection_context.cc:371 - Disconnected 10.16.67.77:17912 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,307 [shard 41] kafka - connection_context.cc:371 - Disconnected 10.16.67.76:44540 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,307 [shard 3] kafka - connection_context.cc:371 - Disconnected 10.16.65.53:37422 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 30] kafka - connection_context.cc:371 - Disconnected 10.16.65.67:58692 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 30] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.65.67:58692 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 31] kafka - connection_context.cc:371 - Disconnected 10.16.67.122:11870 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 59] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.87:14335 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 26] kafka - connection_context.cc:371 - Disconnected 10.16.67.17:29976 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 23] kafka - connection_context.cc:371 - Disconnected 10.16.65.71:60228 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.25.4.11:44364 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.67.72:33098 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 54] kafka - connection_context.cc:371 - Disconnected 10.16.67.33:47058 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 62] kafka - connection_context.cc:371 - Disconnected 10.16.67.83:35654 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 23] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.65.71:60228 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 46] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.72:33098 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 7] kafka - connection_context.cc:371 - Disconnected 10.25.0.71:50464 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 7] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.0.71:50464 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 59] kafka - connection_context.cc:371 - Disconnected 10.16.66.87:14335 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 46] kafka - connection_context.cc:371 - Disconnected 10.16.66.159:61565 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 42] kafka - connection_context.cc:371 - Disconnected 10.16.67.116:25250 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 24] kafka - connection_context.cc:371 - Disconnected 10.16.67.33:46938 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,308 [shard 24] kafka - connection_context.cc:371 - Disconnected 10.16.67.27:28630 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.67.19:39430 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 24] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.33:46938 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 46] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.66.159:61565 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 24] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.27:28630 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 3] kafka - connection_context.cc:371 - Disconnected 10.25.12.68:23496 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 56] kafka - connection_context.cc:371 - Disconnected 10.16.66.191:32808 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 3] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.12.68:23496 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 57] kafka - connection_context.cc:371 - Disconnected 10.16.67.124:21558 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,309 [shard 60] kafka - connection_context.cc:371 - Disconnected 10.16.67.118:28560 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.25.4.11:44364 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 25] kafka - connection_context.cc:371 - Disconnected 10.16.67.77:17912 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 26] kafka - connection_context.cc:371 - Disconnected 10.16.66.114:29008 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 26] kafka - connection_context.cc:371 - Disconnected 10.16.67.17:29976 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 54] kafka - connection_context.cc:371 - Disconnected 10.16.67.33:47058 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 62] kafka - connection_context.cc:371 - Disconnected 10.16.67.83:35654 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,310 [shard 54] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.33:47058 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.67.19:39430 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 59] kafka - connection_context.cc:371 - Disconnected 10.25.4.48:41332 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 60] kafka - connection_context.cc:371 - Disconnected 10.16.67.118:28560 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 57] kafka - connection_context.cc:371 - Disconnected 10.16.67.124:21558 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 57] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.124:21558 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 27] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.19:39430 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 39] kafka - connection_context.cc:371 - Disconnected 10.25.4.11:44364 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,311 [shard 60] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.118:28560 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,312 [shard 39] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.4.11:44364 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,312 [shard 26] kafka - connection_context.cc:371 - Disconnected 10.16.66.114:29008 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,312 [shard 26] kafka - connection_context.cc:371 - Disconnected 10.16.67.17:29976 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,312 [shard 26] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.17:29976 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,312 [shard 25] kafka - connection_context.cc:371 - Disconnected 10.16.67.77:17912 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,313 [shard 49] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.19:39978 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,313 [shard 25] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.16.67.77:17912 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,318 [shard 59] rpc - server.cc:135 - kafka rpc protocol - Disconnected 10.25.4.48:41332 (shutting down, Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,322 [shard 49] kafka - connection_context.cc:371 - Disconnected 10.16.67.19:39978 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,323 [shard 9] kafka - connection_context.cc:371 - Disconnected 10.16.65.52:52438 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,323 [shard 49] kafka - connection_context.cc:371 - Disconnected 10.16.67.19:39978 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,325 [shard 6] kafka - connection_context.cc:371 - Disconnected 10.16.66.191:33636 (Broken pipe) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,325 [shard 59] kafka - connection_context.cc:371 - Disconnected 10.25.4.48:41332 (stream closed) Nov 18 22:40:12 srv-01-08-309.iad1.trmr.io rpk[54634]: INFO 2022-11-18 22:40:12,325 [shard 27] kafka - connection_context.cc:371 - Disconnected 10.16.67.26:44746 (Broken pipe)

Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 115 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450de59 0x450b0e6 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb886302e 0xffffffffb8e000df 0xffffffffb809c8bf 0xffffffffb8ab48be 0xffffffffb8c00d67 0xffffffffb8ac881d 0xffffffffb80fc43a 0xffffffffb80fc483 0xffffffffc0445151 0xffffffffb87db0b0 0xffffffffb87db24d 0xffffffffb84929fb 0xffffffffb8492f15 0xffffffffb84932ef 0xffffffffb849363a 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8e00072 0xffffffffb809c8bf 0xffffffffb8ab487a 0xffffffffb8c00d67 Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb81064f5 0xffffffffc05e3d92 0xffffffffc05e3ee6 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb87db20e 0xffffffffb84929fb 0xffffffffb8492f15 0xffffffffb84932ef 0xffffffffb849363a 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb892da96 0xffffffffb893f779 0xffffffffb8941ae1 0xffffffffb890d827 0xffffffffb890daa6 0xffffffffb890db3f 0xffffffffb890dd39 0xffffffffb890dedd 0xffffffffb890e19a 0xffffffffb886218a 0xffffffffb886238a 0xffffffffb8862afa 0xffffffffc049e3cc 0xffffffffb8862c81 0xffffffffb88631c2 0xffffffffb8e000df 0xffffffffb809c8bf 0xffffffffb8ab487a 0xffffffffb8c00d67 Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8ac87e5 0xffffffffb83ac95e 0xffffffffb83482a0 0xffffffffb8348607 0xffffffffb83486aa 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8250f1b 0xffffffffb82517f8 0xffffffffb83615c5 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb853f00d 0xffffffffb853f0ed 0xffffffffb84b38e7 0xffffffffb84b53df 0xffffffffb84b567e 0xffffffffb848c082 0xffffffffb848daff 0xffffffffb83cf729 0xffffffffb83cfa19 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e4b9d 0xffffffffc05e4cf2 0xffffffffb83b13c6 0xffffffffb83b20ca 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 80 ms on shard 28. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbdea 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8500959 0xffffffffc060c83e 0xffffffffc06050fb 0xffffffffc0605630 0xffffffffc05f27d5 0xffffffffb836aa1e 0xffffffffb836ceff 0xffffffffc05e3f9c 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc0442f5f 0xffffffffc04432e4 0xffffffffc044516d 0xffffffffb87db0b0 0xffffffffb87db24d 0xffffffffb84929fb 0xffffffffb8492f15 0xffffffffb84932ef 0xffffffffb849363a 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8ac87e5 0xffffffffb83ac95e 0xffffffffb83482a0 0xffffffffb8348607 0xffffffffb83486aa 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb854b907 0xffffffffb82b1170 0xffffffffb83163fa 0xffffffffb8318879 0xffffffffb82b0725 0xffffffffb82b080b 0xffffffffb807c7fe 0xffffffffb8ab729c 0xffffffffb8c00b77 Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8493111 0xffffffffb849363a 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc05d5ada 0xffffffffc05f27cb 0xffffffffb836aa1e 0xffffffffb836ae45 0xffffffffb836b417 0xffffffffc05e4331 0xffffffffc05e4b6b 0xffffffffc05e4cf2 0xffffffffb83b13c6 0xffffffffb83b20ca 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc060b3d5 0xffffffffc06050fb 0xffffffffc0605630 0xffffffffc05f27d5 0xffffffffb836aa1e 0xffffffffb836ae45 0xffffffffb836b417 0xffffffffc05e4331 0xffffffffc05e4b6b 0xffffffffc05e4cf2 0xffffffffb83b13c6 0xffffffffb83b20ca 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb828fee5 0xffffffffb8267414 0xffffffffb82674de 0xffffffffb83cfe64 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb809c748 0xffffffffb885fade 0xffffffffc0f7eb6f 0xffffffffc0f807d3 0xffffffffb885f56e 0xffffffffb885fefa 0xffffffffc0fac0da 0xffffffffb885f56e 0xffffffffb885fefa 0xffffffffb886893d 0xffffffffb89115bd 0xffffffffb891279e 0xffffffffb89128ba 0xffffffffb89129da 0xffffffffb8913bfe 0xffffffffb8913fba 0xffffffffb8914275 0xffffffffb8935c97 0xffffffffb893747a 0xffffffffb8937c67 0xffffffffb8920618 0xffffffffb892143e 0xffffffffb8921e8d 0xffffffffb895e303 0xffffffffb8833fc2 0xffffffffb883431c 0xffffffffb8836731 0xffffffffb88367ec 0xffffffffb883685f 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb84936ce 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e4b9d 0xffffffffc05e4cf2 0xffffffffb83b13c6 0xffffffffb83b20ca 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8839b01 0xffffffffb8839fe1 0xffffffffb89213b2 0xffffffffb8921e8d 0xffffffffb895e303 0xffffffffb8833fc2 0xffffffffb883431c 0xffffffffb8836731 0xffffffffb88367ec 0xffffffffb883685f 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb830b56e 0xffffffffb8269987 0xffffffffb82697c4 0xffffffffb848da93 0xffffffffb83cf729 0xffffffffb83cfa19 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8067793 0xffffffffb80675b6 0xffffffffb81606f4 0xffffffffb81607e9 0xffffffffb807fbbb 0xffffffffb807fd5d 0xffffffffb82c152e 0xffffffffb82a819d 0xffffffffb82abdd7 0xffffffffb82b0703 0xffffffffb82b080b 0xffffffffb807c7fe 0xffffffffb8ab729c 0xffffffffb8c00b77 Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc049bc0e 0xffffffffc049e368 0xffffffffb8862c81 0xffffffffb88631c2 0xffffffffb8e000df 0xffffffffb809c8bf 0xffffffffb8ab48be 0xffffffffb8c00d67 0xffffffffb854b907 0xffffffffb82b1170 0xffffffffb83163fa 0xffffffffb8318879 0xffffffffb82b0725 0xffffffffb82b080b 0xffffffffb807c7fe 0xffffffffb8ab729c 0xffffffffb8c00b77 Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 77 ms on shard 59. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x3ededc2 0x3edea12 0x3e7d0cf 0x3e77983 0x3441b97 0x344163a 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8ab77b8 0xffffffffb8ab3126 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb849367e 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb8ac87e5 0xffffffffb83abdb0 0xffffffffb83abfec 0xffffffffb83ac264 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb836ee89 0xffffffffb83b1d8e 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb848e49a 0xffffffffb83cfa56 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc05e40a8 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb854bfdc 0xffffffffb884a559 0xffffffffb884a5e8 0xffffffffb884a8b8 0xffffffffb8922b0c 0xffffffffb8923914 0xffffffffb895e434 0xffffffffb8833c61 0xffffffffb8833cff 0xffffffffb8345c62 0xffffffffb834818b 0xffffffffb8348525 0xffffffffb834857a 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb83098e1 0xffffffffb8269757 0xffffffffb82697c4 0xffffffffc0904bb9 0xffffffffc09064b2 0xffffffffb84a1e79 0xffffffffb8493b20 0xffffffffb8493d9a 0xffffffffb83d0295 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb842fbfb 0xffffffffb8345535 0xffffffffb83480b0 0xffffffffb8348525 0xffffffffb834857a 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc0440106 0xffffffffc044017c 0xffffffffc0442e7f 0xffffffffc04432e4 0xffffffffc044516d 0xffffffffb87db0b0 0xffffffffb87db24d 0xffffffffb84929fb 0xffffffffb8492f15 0xffffffffb84932ef 0xffffffffb849363a 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb849301a 0xffffffffb84932ef 0xffffffffb849363a 0xffffffffb83cf672 0xffffffffb83cfac2 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffb83cf605 0xffffffffb83d022e 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: 0xffffffffffffff80 0xffffffffc090513b 0xffffffffc09064b2 0xffffffffb84a1e79 0xffffffffb8493b20 0xffffffffb8493d9a 0xffffffffb83d0295 0xffffffffb83d0511 0xffffffffc05e3f6a 0xffffffffc05e4135 0xffffffffb83b177f 0xffffffffb83b2027 0xffffffffb83b5338 0xffffffffb83b5b67 0xffffffffb8ab314b 0xffffffffb8c0009b Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 181 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x44fbed5 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 138 ms on shard 28. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450a89b 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 135 ms on shard 59. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x3eded5f 0x3edea12 0x3e7d0cf 0x3e77983 0x3441b97 0x344163a 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 288 ms on shard 27. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450dc0c 0x450b0e6 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: Reactor stalled for 245 ms on shard 28. Backtrace: 0x4ab0f1f 0x4ab0c32 0x42abf 0x450be2a 0x44fbfcf 0x4391bdb 0x4a1b3ac 0x4a2143c 0x4a30c11 0x3e83653 0x3decb62 0x3df481e 0x3dee862 0x3dedc98 0x43c59a9 0x43c56b6 0x43c6174 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f 0x91016 0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: kernel callstack: Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,301 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,322 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,343 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,363 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: WARN 2022-11-18 22:40:02,369 [shard 27] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: -------- Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: WARN 2022-11-18 22:40:02,379 [shard 28] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace: 0x4d1d6c4 0x4a1298b 0x3dfb960 0x4ace59f 0x4ad2277 0x4b15745 0x4a6f10f /opt/redpanda/lib/libc.so.6+0x91016 /opt/redpanda/lib/libc.so.6+0x1166cf Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: -------- Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&), seastar::futurize<cluster::persisted_stm::make_snapshot_in_background()::$_3>::type seastar::future<void>::then_wrapped_nrvo<seastar::future<void>, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)>(seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&&)::'lambda'(seastar::internal::promise_base_with_type<void>&&, seastar::future<void> seastar::future<void>::handle_exception_type<auto ssx::spawn_with_gate_then<cluster::persisted_stm::make_snapshot_in_background()::$_3>(seastar::gate&, cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(seastar::broken_condition_variable const&)>(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)::'lambda'(cluster::persisted_stm::make_snapshot_in_background()::$_3&&)&, seastar::future_state<seastar::internal::monostate>&&), void> Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,383 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,404 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,424 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,444 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,464 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,484 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,504 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,523 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,543 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,562 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,583 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,603 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,623 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,644 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,665 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,685 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,706 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,727 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,747 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,768 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,788 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,810 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,831 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,851 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,872 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,893 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,913 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,934 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,955 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,975 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:02 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:02,997 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,017 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,038 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,059 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,080 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,100 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,122 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,142 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,163 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,184 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,205 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,225 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,246 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,267 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,288 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,309 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,330 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,351 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,371 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,392 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,414 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,434 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,455 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,476 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,497 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,518 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,523 [shard 4] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:14, base_offset:2286935605, committed_offset:2287256646, dirty_offset:2287256646}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rxdata-videoevent/3_1284/2286935605-14-v1.log, (108108551 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rxdata-videoevent/3_1284/2286935605-14-v1.base_index, offsets:{2286935605}, index:{header_bitflags:0, base_offset:{2286935605}, max_offset:{2287256646}, base_timestamp:{timestamp: 1668569508429}, max_timestamp:{timestamp: 1668569993353}, index(3228,3228,3228)}, step:32768, needs_persistence:0}} Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,523 [shard 4] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rxdata-videoevent/3_1284/2286935605-14-v1.log", "/var/lib/redpanda/data/kafka/rxdata-videoevent/3_1284/2286935605-14-v1.base_index"} Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,539 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,553 [shard 4] offset_translator - ntp: {kafka/rxdata-videoevent/3} - offset_translator.cc:277 - prefix_truncate at offset: 2287256646, new state: {base offset/delta: {2287256646}/12, map size: 46, last delta: 57} Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,560 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,580 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,601 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,622 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,643 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,664 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,684 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,705 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,726 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,746 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,767 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,788 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,809 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,829 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,850 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,870 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,890 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,910 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,930 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,949 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,969 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:03 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:03,990 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,011 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,031 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,052 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,073 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,094 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,115 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,136 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,156 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,177 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,198 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,219 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,239 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,260 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,281 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,302 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,323 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,344 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,365 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,385 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,406 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,427 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,448 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,469 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,489 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,510 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,531 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,552 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,572 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,593 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,614 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,635 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,656 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,677 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,697 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,718 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,739 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,760 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,780 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,801 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,822 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,836 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] vote_stm.cc:77 - vote reply from {id: {2}, revision: {1256}} - {term:{70}, target_node_id{id: {3}, revision: {1256}}, vote_granted: 1, log_ok:1} Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,837 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] vote_stm.cc:263 - becoming the leader term:70 Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,837 [shard 28] storage - segment.cc:630 - Creating new segment /var/lib/redpanda/data/kafka/rxdata-impserve/9_1256/2888454580-70-v1.log Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,837 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] vote_stm.cc:77 - vote reply from {id: {0}, revision: {1256}} - {term:{70}, target_node_id{id: {3}, revision: {1256}}, vote_granted: 1, log_ok:1} Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,843 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,864 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,884 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,905 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,926 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,946 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,967 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:04 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:04,988 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,004 [shard 28] raft - [group_id:6940, {kafka/rxdata-impserve/9}] vote_stm.cc:279 - became the leader term: 70 Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,009 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,030 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,051 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,072 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,092 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,113 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,133 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,154 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,175 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,195 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,216 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,236 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,257 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,277 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,297 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,316 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,336 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,355 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,375 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,395 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,415 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,436 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,456 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,477 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,497 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,518 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,534 [shard 58] storage - disk_log_impl.cc:1005 - remove_prefix_full_segments - tombstone & delete segment: {offset_tracker:{term:21, base_offset:493196923, committed_offset:493217920, dirty_offset:493217920}, compacted_segment=0, finished_self_compaction=0, generation={0}, reader={/var/lib/redpanda/data/kafka/rxdata-bidrequest/1_1236/493196923-21-v1.log, (101820322 bytes)}, writer=nullptr, cache={cache_size=0}, compaction_index:nullopt, closed=0, tombstone=0, index={file:/var/lib/redpanda/data/kafka/rxdata-bidrequest/1_1236/493196923-21-v1.base_index, offsets:{493196923}, index:{header_bitflags:0, base_offset:{493196923}, max_offset:{493217920}, base_timestamp:{timestamp: 1668569882158}, max_timestamp:{timestamp: 1668569997170}, index(2826,2826,2826)}, step:32768, needs_persistence:0}} Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,534 [shard 58] storage - segment.cc:112 - removing: {"/var/lib/redpanda/data/kafka/rxdata-bidrequest/1_1236/493196923-21-v1.log", "/var/lib/redpanda/data/kafka/rxdata-bidrequest/1_1236/493196923-21-v1.base_index"} Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,538 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,559 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,567 [shard 58] offset_translator - ntp: {kafka/rxdata-bidrequest/1} - offset_translator.cc:277 - prefix_truncate at offset: 493217920, new state: {base offset/delta: {493217920}/21, map size: 63, last delta: 83} Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,579 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,600 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,621 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,642 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,663 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,683 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,704 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,725 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,745 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,766 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,786 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,807 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,828 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,849 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,869 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,890 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,910 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,931 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,952 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,973 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:05 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:05,993 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,014 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,034 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,055 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,075 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,096 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,116 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,136 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,157 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,178 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,199 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,219 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,240 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,260 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,281 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,302 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,323 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,343 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,364 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,384 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc) Nov 18 22:40:06 srv-01-04-404.iad1.trmr.io rpk[10694]: INFO 2022-11-18 22:40:06,405 [shard 59] cluster - state_machine.cc:141 - State machine for ntp={kafka/rxdata-bidrequest/5} handles std::bad_alloc (std::bad_alloc)

redpanda-data / redpanda

Cannot Start Redpanda After Upgrading from 22.2.7 to 22.3.2 (or 22.3.1) #7343

Version & Environment

What went wrong?

What should have happened instead?

How to reproduce the issue?

Additional information