search

Particular / NServiceBus

Build, version, and monitor better microservices with the most powerful service platform for .NET
https://particular.net/nservicebus/
Other
2.08k stars 647 forks source link

Support outbox record expiration / TTL in Outbox #3043

Open ramonsmits opened 8 years ago

ramonsmits commented 8 years ago

While discussing TTL (Time To Live / Time To Be Received) for the SQL transport I was thinking about MSMQ its TTRQ attribute. It does not make sense to have this in the SQL transport but it does make sense for the Outbox. The Outbox should be able to purge items that have an expired TTL is that is set on the message.

If a queue is offline then the outbox will eventually start purging message that have a TTL set.

TTL would be set on the outbox configuration and not on the message or transport configuration.

Outbox TTL does not have a relation with message TTBR, TTRQ or TTL.

//cc: @mauroservienti @tmasternak

mauroservienti commented 8 years ago

I like this, also it is a good way to try to reach, as much as we can, feature parity across transports

ramonsmits commented 8 years ago

The TimeToBeReceived is currently being removed by #2886 due to a MSMQ behavior that we do not check for.

Names like TTRQ and TTBR as MSMQ specific but RabbitMQ has support for TTL but at the queue level and Azure Storage Queues has an implicit TTL of 7 days but can be set to be less.

Outbox TTL is to prevent to keep message in the outbox for endpoints that are offline for a long time.

So after this I'm renaming the issue to prevent any relation with transport specific TTL's

ramonsmits commented 8 years ago

@mauroservienti @tmasternak I've edited the description.

@andreasohlund Would this be a feature for consideration?

tmasternak commented 8 years ago

Just a comment on TimeToBeRecieved by @ramonsmits. It looks that TTBR will be un-supported only with MSMQ+transactions configuration.

andreasohlund commented 6 years ago

@ramonsmits if I get you what you're saying is to make sure that the persisters is aware of the message TTL should one exist so that they can optimize their cleanup?

This is already possible today since the TTL, if set, is available as a header and it could be pulled out of the context passed to IOutboxStorage.Store

Eg. the InMemoryPersister could pull it out here:

https://github.com/Particular/NServiceBus/blob/develop/src/NServiceBus.Core/Persistence/InMemory/Outbox/InMemoryOutboxStorage.cs#L31

Given the small use we've heard of the TTL since it doesn't work as expected on most transports I'd say that knowing that it can be done is enough for now so lets close this issue?

timbussmann commented 5 years ago

is this still relevant? Afaik the outbox persisters can define their own custom TTL configuration. Is this something we want to pull into the seam API? If so, is that already covered by architecture?