High frequency of queries

I am noticing a very high frequency of database queries while just using 1 worker. The database query fired very often (multiple times even on a single millisecond) is:

         SET NOCOUNT ON

    ;WITH TopCTE AS (
        SELECT  TOP 1
                [id],
                [headers],
                [body]
        FROM    [dbo].[WorklistApiInputQueue] M WITH (ROWLOCK, READPAST, READCOMMITTEDLOCK)
        WHERE   
                M.[visible] < sysdatetimeoffset()
        AND     M.[expiration] > sysdatetimeoffset()
        ORDER
        BY      [priority] DESC,
                [visible] ASC,
                [id] ASC
    )
    DELETE  FROM TopCTE
    OUTPUT  deleted.[id] as [id],
            deleted.[headers] as [headers],
            deleted.[body] as [body]

    SET NOCOUNT OFF

In my log I see only one worker started up:

2021-12-01 16:18:58.874 +01:00 [INF] Database already contains a table named "[dbo].[WorklistApiInputQueue]" - will not create anything 2021-12-01 16:18:58.904 +01:00 [INF] Starting periodic task "ExpiredMessagesCleanup" with interval 00:00:20 2021-12-01 16:18:58.905 +01:00 [INF] Starting periodic task "CleanupTrackedErrors" with interval 00:00:10 2021-12-01 16:18:58.907 +01:00 [INF] Database already contains a table named "[dbo].[error]" - will not create anything 2021-12-01 16:18:58.943 +01:00 [INF] Bus "Rebus 1" setting number of workers to 1 2021-12-01 16:18:58.944 +01:00 [INF] Bus "Rebus 1" started

However, the SQL trace shows many of those queries:

My code to start Rebus is:

                // Use a single Rebus bus instance, do not recreate it every 10 seconds
                using (var activator = new BuiltinHandlerActivator())
                {
                    var options = new SqlServerTransportOptions(_configuration.GetConnectionString("RebusDb"))
                           .AsOneWayClient(); //one-way, only send
                    Configure.With(activator)
                        .Logging(l => l.MicrosoftExtensionsLogging(_logger))
                        .Serialization(s=>s.UseNewtonsoftJson(JsonInteroperabilityMode.PureJson))
                        .Transport(t => t.UseSqlServer(options, "WorklistApiInputQueue")) //note: we don't use the input queue other than for subscriptions
                        .Subscriptions(s => s.StoreInSqlServer(_configuration.GetConnectionString("RebusDb"), "Subscriptions", isCentralized: true))
                        .Start();

Am I doing something wrong? Or is this high load of queries expected?

Testing with:

    <PackageReference Include="Rebus" Version="6.6.1" />
    <PackageReference Include="Rebus.Microsoft.Extensions.Logging" Version="2.0.0" />
    <PackageReference Include="Rebus.SqlServer" Version="7.2.0" />

Rebus defaults to run with a parallelism of 5, which results in 5 more or less simultaneous queries when you start the bus.

It should slow down after a short while through, as soon as it detects that it's running idle, and the backoff behaviour kicks in.

If you don't need parallel processing of messages, you can reduce the parallelism to 1:

services.AddRebus(
    configure => configure
        .(...)
        .Options(o => o.SetMaxParallelism(1))
);

If you want to reduce the number of queries even more, you can configure a custom backoff behavior:

services.AddRebus(
    configure => configure
        .(...)
        .Options(o => {
            o.SetBackoffTimes(
            TimeSpan.FromSeconds(1), 
            TimeSpan.FromSeconds(3), 
            TimeSpan.FromSeconds(5),
            TimeSpan.FromSeconds(10));
       })
);

at the expense of slower reaction time when receiving a message after having been idle for a while.

I hope that makes sense to you 🙂

rebus-org / Rebus.SqlServer

High frequency of queries #93