Message too large for batch and then not sent

[peterhutgasunie]

Symptoms

Message larger than 1024KB sent from handler is not sent, despite logging that the message would be sent individually (outside a batch).

Who's affected

All users of NServiceBus.Transport.AzureServiceBus 4.2.0 sending messages larger than 1024KB are affected.

Root cause

An async/await bug caused code to run out of logical order, resulting in dispatching an empty list of too-large messages before the too-large messages are identified.

Original bug report

Describe the bug

Description

Logging showed that: "Message '4aae1ee3-7fca-432f-9a35-b1f100ae9abe' is too large for the batch '0' and will be sent individually to destination sbq-acc-ctb-shared-external-meteringworker." But then nothing happened: the message is not sent.

Expected behavior

Message is sent and either the message is placed in the Azure Service Bus OR an exception is thrown if the message is too large for the queue.

Actual behavior

Nothing happens after reporting that it is too large for the batch.

Versions

NServiceBus 9.1.1 Azure Service Bus transport 4.2.0

Steps to reproduce

Send message that is larger than batch size (1024KB). Message size configured on the queue does not matter.

Relevant log output

On environment A:
"Message '4aae1ee3-7fca-432f-9a35-b1f100ae9abe' is too large for the batch '0' and will be sent individually to destination sbq-acc-ctb-shared-external-meteringworker."
and then nothing.

Additional Information

Workarounds

None.

Possible solutions

Looking at the code I suspect an issue in MessageDispatcher. https://github.com/Particular/NServiceBus.Transport.AzureServiceBus/blob/master/src/Transport/Sending/MessageDispatcher.cs#L134

On line 134 a Task to execute DispatchBatchForDestination is added to dispatchTasks and passes the list messagesTooLargeToBeBatched into it. Then on line 136 that list is iterated immediately, but I think the DispatchBatchForDestination may not have been executed then yet, so if a message is too large it will only be added to the list after the iteration on line 136, with an empty list, has already happened.

Could this be what is happening? I also see errors that the message is sent and then fails because the message is too large for the configured queue. Not sure how that can happen, as I would suspect it always loses the messages, so maybe something is missing in my logic.

Additional information

[DavidBoike]

Will be released as 4.2.1.

[DavidBoike]

@peterhutgasunie 4.2.1 has been released.

https://discuss.particular.net/t/nservicebus-transport-azureservicebus-4-2-1-critical-patch-release-available/4121

[danielmarbach]

Following through with a simplification of the dispatcher https://github.com/Particular/NServiceBus.Transport.AzureServiceBus/pull/1052