search

Particular / NServiceBus.Persistence.Sql

Native SQL Persistence for NServiceBus
https://docs.particular.net/persistence/sql/
Other
36 stars 27 forks source link

Cannot customize outbox clean up features #333

Closed jamesbarkerum closed 3 years ago

jamesbarkerum commented 5 years ago

We have many many nodes in our infrastructure and we are migrating off of RavenDb to using this package. Previously there were two configuration options that we had used to prevent NServiceBus hammering raven from constant querying from every node (we could set the timespan to -1).

As it's also a high throughput system the hard coded delete parameter of 7 days is too long to cleanup the outbox table so we've always had a tighter cleanup procedure

These are the two configuration options not available on this package: configuration.SetTimeToKeepDeduplicationData(settings.TimeToKeepDeduplicationTimeSpan); configuration.SetFrequencyToRunDeduplicationDataCleanup(settings.FrequencyToRunDeduplicationDataCleanupTimeSpan);

At the moment they are hard coded in SqlOutboxFeature.cs context.RegisterStartupTask(b => new OutboxCleaner(outboxPersister.RemoveEntriesOlderThan, b.Build<CriticalError>().Raise, TimeSpan.FromDays(7), TimeSpan.FromMinutes(1), new AsyncTimer()));

There is no reason to have every single node pinging the same db every minute for the same cleanup procedure.

ramonsmits commented 5 years ago

Check this doco https://docs.particular.net/persistence/sql/outbox

In a scaled out environment a better option is to have a cleanup task running as a SQL server task instead of having the endpoint instances invoke the cleanup.

ramonsmits commented 5 years ago

Cleanup is there but maybe you are using an older version

https://github.com/Particular/NServiceBus.Persistence.Sql/blob/develop/src/SqlPersistence/Outbox/SqlOutboxFeature.cs

jamesbarkerum commented 5 years ago

Hi Ramon,

Thanks. At the moment we are on NSB 6.x and SQL 3.x (postgres backend) as we have not started our move to .Net Standard yet in our wider code base. Is there a way to get a backport for the v3 package?

jamesbarkerum commented 5 years ago

Or if there is another way to disable it in older package so we can create our own offline task?

ramonsmits commented 5 years ago

@jamesbarkerum Our v7 package also supports .net full framework. It is not required to transition to .net core. There should be no big issue preventing you to update to NServiceBus 7.

mikeminutillo commented 3 years ago

@jamesbarkerum we are closing this issue due to inactivity. If you still require assistance, please let us know.