Azure / azure-service-bus

☁️ Azure Service Bus service issue tracking and samples
https://azure.microsoft.com/services/service-bus
MIT License
585 stars 781 forks source link

[Service Bus] Max time of message lock duration #470

Open DzcleisureCN opened 2 years ago

DzcleisureCN commented 2 years ago

Description

As AWS(SQS) Max time of the message lock duration is more than 2 hours but service bus is only 5 minutes. When we want to transfer from SQS to the service bus it is hard. As I know service bus can auto-renew but this needs a client base receiver, but usually the process message and receive message is not a client, just use the ticket to process, it needs a service bus auto-renew to enlarge the message lock duration

Actual Behavior

  1. Max time of the message lock duration is 5 minutes

Expected Behavior

  1. Max time of the message lock duration is more than 15 minutes
SeanFeldman commented 2 years ago

As I know service bus can auto-renew but this needs a client base receiver, but usually the process message and receive message is not a client, just use the ticket to process, it needs a service bus auto-renew to enlarge the message lock duration

I'm not sure about that. Messages are received and processed by some sort of client. So you can extend the lock for hours and hours. It's not guaranteed that renewing will work all the time as this is not a server-side operation.

@EldertGrootenboer might help here a bit more.

PS: if you want a simple service like SQS, Azure Storage Queues is the equivalent and can lock the message for a custom time. But ASQ is not as nice as ASB 🙂

DzcleisureCN commented 2 years ago

Hi @SeanFeldman Thanks for your reply we do not use ASQ because it does not support long polling. We just want ASB Max time of the message lock duration more than 1 hour and do not need a client to auto-renew. After we received the message will close the receiver immediately, and the message send to another process to handle. Thanks~

EldertGrootenboer commented 2 years ago

As Sean mentioned, you should be able to extend the lock on the same client you use to receive the message. If this doesn't work for you, can you explain a bit more about the process you are using?

DzcleisureCN commented 2 years ago

@EldertGrootenboer Thanks for your response. Our use case is that there is a module Apiserver in Cloud and another module Worker in the enterprise data center. The worker will request Apiserver to get the message then the APIserver will request a service bus and return the message to Worker. The worker will analyze the message(which may need more than 15 minutes) and then send the result to API Server and the API server will delete the message from the service bus.

Before Apiserver build on AWS and AWS SQS had a ticket after receiving the message from SQS, using this ticket can delete the message anywhere and AWS SQS can set the lock duration longer than 5 minutes.

Now we change the API server to Azure and use the service bus. When the worker requests the API server to get the message then the API server will request a service bus to get the message returned to the worker, we want the API server will do not need to keep the client connection and auto-renew, just get the message and finish. And the message will not return to the service bus in about 15 minutes.

In another word we want configure the server bus max lock duration is 15 minutes, and service side auto-renew to extend the lock duration from 5 minutes to 15 minutes, and do not need client to keep renew after review the message

EldertGrootenboer commented 2 years ago

@DzcleisureCN we are working on a new feature (timelines for when this will be released are not available yet) which will remove the need to extend the lock, however this will still expect the client to keep the connection open. It can recover from connection loss while retaining the message state, but we do expect the client to keep the connection open in general, or to reconnect as soon as possible.

DzcleisureCN commented 2 years ago

@EldertGrootenboer Thanks a lot for your reply. But would I ask why after receiving a message from a service bus also need to keep the connection? Why not like AWS, after receiving the message from SQS, do not need to keep the connection. Using the ticket of the message, any other program can operate the message. My understanding is that lock of SQS is a service-side operation, but the lock of ASB is the client-side operation? Have any plans to enhance it? Because after receiving the message, keeping a connection is more inconvenient than AWS SQS.

SeanFeldman commented 2 years ago

@DzcleisureCN ASB is not the same as SQS. If you're looking for an SQS-like service, ASQ (Azure Storage Queues) is what you need. Not to be rude but this has already been pointed out earlier. Perhaps taking time to read up on ASQ vs ASB would help to cut down the number of questions that have been answered and documented. Cheers!

DzcleisureCN commented 2 years ago

@SeanFeldman Thanks for your reply. Before replying already explain why do not use ASQ, the 2 most important points are from your document. ASQ misses many features than SQS, from a feature point, ASB is SQS-like service, that is why we ask this question.

image

SeanFeldman commented 2 years ago

Why not like AWS, after receiving the message from SQS, do not need to keep the connection.

ASB is a long-polling service. That's how it's designed. ASQ and SQS are HTTP-based services and do not keep a connection. Instead, those services poll for messages. ASB might resemble SQS when it comes to some features but it's not an equivalent. RabbitMQ is more aligned with ASB than SQS.

EldertGrootenboer commented 1 year ago

Thank you for your feedback on this item. We are currently actively working on this feature, however we don't have a timeline to share yet around its release.

EldertGrootenboer commented 6 months ago

We have brought this item in our current planning, and should be able to provide an update in the coming months.