Given I am producing, consuming, and reporting metadata on a topic managed by 3 broker nodes
And the Group Coordinator broker is killed (cleanly)
Then the producer, consumer, and reporter will all fatally crash and not recover.
The output of this metadata error (from the producer, for example):
Relevant exn:
Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
2018-04-18 22:19:57:6517|WARN|Kafunk.Resource|failed_to_open_resource|type=Socket[127.0.0.1:9092] version=2 error="System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: Connection refused
--- End of inner exception stack trace ---
at Kafunk.Prelude+ResultModule.throwMap[e,a,a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x0001c] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Faults+retryResultThrow@432[e,a,a].Invoke (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.AsyncEx+AsyncModule+map@147-2[a,b].Invoke (a x) [0x00006] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+args@506-1[a,b].Invoke (a a) [0x00015] in <5a953878dff9fae1a74503837838955a>:0
---> (Inner Exception #0) System.Net.Sockets.SocketException (0x80004005): Connection refused<---
---> (Inner Exception #1) System.Net.Sockets.SocketException (0x80004005): Connection refused<---
"
2018-04-18 22:19:59:6557|WARN|Kafunk.Conn|handling_broker_chan_error|node_id=-2 endpoint=localhost:9092 req=Metadata Kafunk.Protocol+MetadataRequest critical=true errors=[[chan_exn|error="System.AggregateException: One or more errors occurred. ---> Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: Connection refused
--- End of inner exception stack trace ---
at Kafunk.Prelude+ResultModule.throwMap[e,a,a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x0001c] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Faults+retryResultThrow@432[e,a,a].Invoke (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.AsyncEx+AsyncModule+map@147-2[a,b].Invoke (a x) [0x00006] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+args@506-1[a,b].Invoke (a a) [0x00015] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
at Kafunk.Resource+Resource+go@220-25[b,r].Invoke (System.Exception _arg8) [0x0003a] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
---> (Inner Exception #0) Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: Connection refused
--- End of inner exception stack trace ---
at Kafunk.Prelude+ResultModule.throwMap[e,a,a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] f, Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x0001c] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Faults+retryResultThrow@432[e,a,a].Invoke (Microsoft.FSharp.Core.FSharpChoice`2[T1,T2] r) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.AsyncEx+AsyncModule+map@147-2[a,b].Invoke (a x) [0x00006] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+args@506-1[a,b].Invoke (a a) [0x00015] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---
"]] conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6607|INFO|Kafunk.Conn|connecting_to_bootstrap_brokers|brokers=[kafka://localhost:9092/; kafka://localhost:9093/; kafka://localhost:9094/] attempt=1 conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6620|INFO|Kafunk.Conn|discovered_dns|host=localhost ips=[127.0.0.1] conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6638|INFO|Kafunk.Conn|discovered_dns|host=localhost ips=[127.0.0.1] conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6639|INFO|Kafunk.Chan|tcp_connecting|remote_endpoint=127.0.0.1:9093 conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6643|INFO|Kafunk.Chan|tcp_connected|remote_endpoint=127.0.0.1:9093 local_endpoint=127.0.0.1:51619 conn_id=64049b94759e4e5eaa7c8d8cc6d9f818
2018-04-18 22:19:59:6734|WARN|Kafunk.Resource|failed_to_open_resource|type=ProducerState[absurd-gina1-topic] version=2 error="System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
---> (Inner Exception #0) Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---
"
2018-04-18 22:19:59:6748|ERROR|Producer.fsx|produce_error|Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Kafunk.Resource+Resource+go@220-25[b,r].Invoke (System.Exception _arg8) [0x0003a] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
2018-04-18 22:19:59:6771|ERROR|Producer.fsx|produce_error|System.AggregateException: One or more errors occurred. ---> Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
at Kafunk.Resource+Resource+go@220-25[b,r].Invoke (System.Exception _arg8) [0x0003a] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
---> (Inner Exception #0) Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Kafunk.Resource+Open@71-1[r].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00119] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---
2018-04-18 22:19:59:6772|ERROR|Producer.fsx|System.AggregateException: One or more errors occurred. ---> Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at FSI_0002+go@175-23.Invoke (System.Exception _arg9) [0x00030] in <40a4ede61ca94ec1939935132ae0470a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <e36d106f4822473d8910305e5d059e11>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl.commit[a] (Microsoft.FSharp.Control.AsyncBuilderImpl+AsyncImplResult`1[T] res) [0x0002c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronouslyInCurrentThread[a] (System.Threading.CancellationToken token, Microsoft.FSharp.Control.FSharpAsync`1[T] computation) [0x00028] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronously[a] (System.Threading.CancellationToken token, Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] timeout) [0x00013] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T] (Microsoft.FSharp.Control.FSharpAsync`1[T] computation, Microsoft.FSharp.Core.FSharpOption`1[T] timeout, Microsoft.FSharp.Core.FSharpOption`1[T] cancellationToken) [0x00070] in <5a953878dff9fae1a74503837838955a>:0
at <StartupCode$FSI_0002>.$FSI_0002.main@ () [0x00392] in <40a4ede61ca94ec1939935132ae0470a>:0
---> (Inner Exception #0) Kafunk.Resource+FaultedResourceException: The resource faulted. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Kafunk.Tcp+ResponseDecodeException: There was an error decoding the raw TCP response. ---> System.Exception: Unsupported Broker Format for Metadata Api Response Version: 5
at Kafunk.Protocol+Read@848.Invoke (System.String message) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Core.PrintfImpl+SmallStringPrintfEnv`1[TResult].Finish () [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.PrintfImpl+FinalFastEnd1@238[TState,TResidue,TResult,A].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] env, A a) [0x0002a] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Core.OptimizedClosures+Invoke@2806[T2,TResult,T1].Invoke (T2 u) [0x00000] in <5a953878dff9fae1a74503837838955a>:0
at Kafunk.Protocol+Broker.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00023] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+brokers@950.Invoke (Kafunk.BinaryZipper b) [0x00001] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.BinaryZipper.ReadArray[a] (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] read) [0x00019] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+MetadataResponse.Read (System.Int16 ver, Kafunk.BinaryZipper buf) [0x00026] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Protocol+ResponseMessage.Read (Kafunk.Protocol+ApiKey apiKey, System.Int16 apiVer, Kafunk.BinaryZipper buf) [0x0011b] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.ChanModule+decode@269-1[a].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00025] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Kafunk.Tcp+ReqRepSession`3[a,b,s].Demux (System.ArraySegment`1[T] data) [0x0004d] in <5ad669ac6b32e983a7450383ac69d65a>:0
--- End of inner exception stack trace ---
at <StartupCode$Kafunk>.$Kafka+clo@803-50.Invoke (System.Exception _arg19) [0x00060] in <5ad669ac6b32e983a7450383ac69d65a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at FSI_0002+go@175-23.Invoke (System.Exception _arg9) [0x00030] in <40a4ede61ca94ec1939935132ae0470a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithExnA@564[a].Invoke (System.Runtime.ExceptionServices.ExceptionDispatchInfo edi) [0x0000c] in <5a953878dff9fae1a74503837838955a>:0
at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@522[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00051] in <5a953878dff9fae1a74503837838955a>:0 <---
2018-04-18 22:19:59:6801|INFO|Producer.fsx|producer_run_completed|messages=21200 missing=978800 batch_size=100 message_size=10 parallelism=1 elapsed_sec=124.522873 rate_per_sec=170.249847 MB=10
2018-04-18 22:20:00:1615|INFO|Producer.fsx|last_sec=0 | avg_sec=170 | tp50_sec=200 | tp90_sec=0 | tp99_sec=0 | max_sec=200 | count=21200
2018-04-18 22:20:00:1904|INFO|Producer.fsx|completed=21200 elapsed_sec=124.522873 MB=0 offsets=[]
Using Kafka 0.11.0.1:
Given I am producing, consuming, and reporting metadata on a topic managed by 3 broker nodes And the Group Coordinator broker is killed (cleanly) Then the producer, consumer, and reporter will all fatally crash and not recover.
The output of this metadata error (from the producer, for example):
Relevant exn: