Particular / TimeoutMigrationTool

Other
4 stars 1 forks source link

Support for Azure Service Bus #494

Open robvanpamel opened 1 year ago

robvanpamel commented 1 year ago

For a project where I'm working on, we are migrating our NServicebus Transport from SQL Server towards Azure Service Bus. Would it be possible to use this tool to move the "delayed messages" which are still stuck at SQL?

I notice that Azure Service Bus is not listed in the supported transport, but there might be another 'creative' way. Or is the creative way to create a PR that adds ASB support?

robvanpamel commented 1 year ago

I took a closer look at the code, and it is possible to add a AzureServiceBus Target for the migrationtool. So, I started creating one. I do wonder what the exact purpose of the stage is, and how this would be achieved with Azure Service Bus. It can only be delivered to 1 queue...

    public interface IEndpointTargetBatchMigrator : IAsyncDisposable
    {
        /// <summary>
        /// Adds the provided set of timeouts to the staging queue for timeouts
        /// </summary>
        ValueTask<int> StageBatch(IReadOnlyList<TimeoutData> timeouts, int batchNumber);

        /// <summary>
        /// Moves the specified batch from the staging queue to the final delayed delivery queue
        /// </summary>
        ValueTask<int> CompleteBatch(int batchNumber);
    }

Looking at the MSMQ implementation, there is a 'pump' in between, but I don't see the added value (yet). Would it be possible to explain the purpose of staging it first?

danielmarbach commented 1 year ago

@robvanpamel I have not been deeply involved in the design of the timeout migration tool, so take my input with a grain of salt. As far as I recall, the staging approach is influenced by the design of bulk retries used in ServiceControl to reduce the chances of more than once delivery for transports that don't support SendsWithAtomicReceive.

Another reason is that we can't assume transactions span source and target, and hence need to be extra careful to have an algorithm in place that doesn't lose timeouts.