DotNetCore.CAP.Kafka consumer vulnerable to headerless messages

[Ekkeir]

When testing DotNetCore.CAP.Kafka in various scenarios I came across a scenario where consumer ends up endlessly trying to process the same message.

Scenario:

• Set up CAP with Kafka
• Send some proper messages to confirm that the consumer receives messages
• Manually send a Kafka message to the same topic, but without CAP headers

Expected behaviour:

• Either retry policy is applied to message processing or the message is immediately sent to DLQ/ignored with an error message.

Observed behavior:

• The consumer ends up in an endless cycle of trying to consume the message with the following exception observed by DotNetCore.CAP.Internal.ConsumerRegister:

An exception occurred when process received message. Message:'{0}'.

System.Collections.Generic.KeyNotFoundException: The given key 'cap-msg-id' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at DotNetCore.CAP.Internal.ConsumerRegister.<>c__DisplayClass22_0.<<RegisterMessageProcessor>b__0>d.MoveNext()

"0":"DotNetCore.CAP.Messages.TransportMessage"

[yang-xiaodong]

This is the design, CAP is the necessary header information needed to recognize external messages, look up the docs

https://cap.dotnetcore.xyz/user-guide/en/cap/messaging/#heterogeneous-system-integration

[Ekkeir]

I see, so the expectation is that we setup CustomHeadersBuilder deleagte for headerless messages. Thank you.