Protocol error negotiating CBS claim

[jburhenn]

Expected Behavior

The azure-amqp-common-go cbs.NegotiateClaim call successfully negotiates the CBS claim when connecting to an Azure IoT Hub.

Actual Behavior

With some of our hubs the cbs.NegotiateClaim call is erroring out with protocol error: received flow without next-incoming-id after session established. We are using chirpstack-network-server to connect to Azure IoT Hub. It is using azure-amqp-common-go v1.1.4 to connect.

The strange thing is the same code works with some of our hubs but not others. Is this a known issue that later versions of azure-amqp-common-go fix? Or is there some setting that needs to be configured in the IoT Hub to prevent this from happening?

Below is debug output from the underlying pack.ag/amqp package showing some TX/RX frames. It appears to receive a nil value for NextIncomingID which results in the error are seeing. I'm not familiar enough with AMQP to know if that is an error that would be caused by a server side issue, or if there would need to be some change on the client side code to fix this.

16:57:15.364382 TX: Begin{RemoteChannel: 0, NextOutgoingID: 0, IncomingWindow: 100, OutgoingWindow: 100, HandleMax: 4294967295, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.412592 TX: Begin{RemoteChannel: 0, NextOutgoingID: 0, IncomingWindow: 100, OutgoingWindow: 100, HandleMax: 4294967295, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.426561 RX: Begin{RemoteChannel: 0, NextOutgoingID: 1, IncomingWindow: 4294967295, OutgoingWindow: 100, HandleMax: 4294967295, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.426665 TX: Attach{Name: L-cTAXLOyxlAOfA6EurUVtCOQu5oCag-yjx6-hNfCT-7b837ef28BQ, Handle: 0, Role: Sender, SenderSettleMode: <nil>, ReceiverSettleMode: <nil>, Source: <nil>, Target: source{Address: $cbs, Durable: 0, ExpiryPolicy: , Timeout: 0, Dynamic: false, DynamicNodeProperties: map[], Capabilities: []}, Unsettled: map[], IncompleteUnsettled: false, InitialDeliveryCount: 0, MaxMessageSize: 0, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.480834 RX: Begin{RemoteChannel: 0, NextOutgoingID: 1, IncomingWindow: 5000, OutgoingWindow: 100, HandleMax: 255, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.480928 TX: Attach{Name: zmoI64EQc9nqHUQFt2bKT0stoh4leXB9_zVIREhtH46qsdjiomn986ls, Handle: 0, Role: Sender, SenderSettleMode: <nil>, ReceiverSettleMode: <nil>, Source: <nil>, Target: source{Address: $cbs, Durable: 0, ExpiryPolicy: , Timeout: 0, Dynamic: false, DynamicNodeProperties: map[], Capabilities: []}, Unsettled: map[], IncompleteUnsettled: false, InitialDeliveryCount: 0, MaxMessageSize: 0, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.490770 RX(Session): Attach{Name: L-cTAXLOyxlAOfA6EurUVtCOQu5oCag-yjx6-hNfCT-7b837ef28BQ, Handle: 0, Role: Receiver, SenderSettleMode: mixed, ReceiverSettleMode: first, Source: <nil>, Target: source{Address: $cbs, Durable: 0, ExpiryPolicy: session-end, Timeout: 0, Dynamic: false, DynamicNodeProperties: map[], Capabilities: []}, Unsettled: map[], IncompleteUnsettled: false, InitialDeliveryCount: 0, MaxMessageSize: 1048576, OfferedCapabilities: [], DesiredCapabilities: [], Properties: map[]}
16:57:15.490825 RX(Session): Flow{NextIncomingID: <nil>, IncomingWindow: 4294967295, NextOutgoingID: 0, OutgoingWindow: 100, Handle: 0, DeliveryCount: 0, LinkCredit: 50, Available: <nil>, Drain: false, Echo: false, Properties: map[]}

Environment

• OS: Ubuntu 20.04.4
• Go version: 1.19.2
• Version of Library: 1.1.4

[asanchezdelc]

It seems that this is related to some sort of azure subscription settings? In one subscription this library works fine with existing and new IoT Hubs, but in other subscription we get the above error.

[nparikhqmc]

Hello, sorry to revive a year old bug. I believe the issue is a difference in the between Azure IoT Hub GWv1 and GWv2. If you click on "Properties" in your IoT Hub, then scroll down to "Features", on the Hub that isn't working, you'll probably see GWv2, and in the one that is working, you won't see this.

It seems that when a hub is created, it's created in v1, then upgraded at a random time in the future to v2. I see that you had posted this in the Chirpstack forums, and it's the exact problem I'm having with Chirpstack in a production instance. I'm working with MS to see if i can have my hub downgraded back to v1 and ensure that it doesn't get upgraded again, but in the interim i had to create a new hub and am just hoping that they dont upgrade it again any time soon. It's frustrating that they would preform a breaking change with no consent or option to prevent it.

I would really appreciate if someone is able to jump on this one.

[asanchezdelc]

@nparikhqmc Thanks heads up. We were able to bypass this issue with updated amqp libs. We'll submit a PR to the Chirpstack repo in the future.

[nparikhqmc]

Thanks for the feedback. Microsoft was able to downgrade our hub back to v1, and we're back online again, but would be nice to have something a little more resilient and compatible with both hub versions in case this happens again.