search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.57k stars 4.82k forks source link

Change task scheduler used by "MessageReceivePump" #10300

Closed dbeavon closed 3 years ago

dbeavon commented 4 years ago

Is there a way to get a message receiver's message pump to use a custom TaskScheduler?

Environment:

I'm assuming I would have to write my own pump for that?

I investigated the code and it looks like the message pump relies on the current TaskScheduler via Microsoft.Azure.ServiceBus.Primitives.TaskExtensionHelper.

I was hoping that I could configure the service bus to deliver messages asynchronously using the Task Factory in Nito.AsyncEx's AsyncContextThread (in Nito.AsyncEx.Factory).

ghost commented 4 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

ghost commented 4 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

jsquire commented 4 years ago

//cc: @nemakam //fyi: @JoshLove-msft , @ShivangiReja

nemakam commented 4 years ago

@dbeavon We do not plan to have that feature in the existing Microsoft.Azure.ServiceBus library. However, there's a new library in pipeline, and @JoshLove-msft / @ShivangiReja could weigh in on this for the new library. Meanwhile, you could write your pump based off our pump if that helps.

For our understanding, what are you looking for with the custom scheduler?

dbeavon commented 4 years ago

@nemakam Thanks for the confirmation. I'm not overly familiar with this library yet and I appreciate your feedback and pointers. (... you are saving me from many hours of googling and research... I already wasted a bit of time by starting off with the wrong version of the service bus client... WindowsAzure.ServiceBus vs Microsoft.Azure.ServiceBus. That forced me to do some minor refactoring, when I saw my mistake).

For our understanding, what are you looking for with the custom scheduler?

I'd like the service bus to deliver messages asynchronously using the Task Factory in Nito.AsyncEx's AsyncContextThread (in Nito.AsyncEx.Factory).

I do NOT want the pump to be pushing messages out to arbitrary threads in the thread pool. I'd want to keep things simple by having messages delivered into a single-threaded AsyncContextThread that has its own synchronization context.

Based on my experience with message consumption, the beginning and ending portions of most processing operations can be serialized into a single thread. However if developers want to push some select portion of the work to the thread pool, then we should be the ones to determine the timing of this for ourselves (ie. with await Task.Run(... the longer-running portion of work ...)).

By using AsyncContextThread, it also avoids some underlying technical problems in certain .Net libraries. ... For example just yesterday I was struggling with TransactionScope, and with a bunch of refactoring work that would become necessary to support async Tasks that are being scheduled in the thread pool. In the TransactionScope scenario there is a funky enumeration that you are forced to start using, called TransactionScopeAsyncFlowOption.Enabled. But I've found that many of these types of concerns are avoided altogether by scheduling all tasks on the AsyncContextThread scheduler. Legacy code "just works" without having to refactor it and re-test it. We still get all the benefits of async programming, but it allows our legacy .Net code to remain isolated on a single thread.

FYI, AsyncContextThread has been a critical component in a lots of my async solutions (running in Windows Services or console applications). I wish something like this was already part of the .Net platforms (for discussion purpose if nothing else). I don't want to feel like I'm referring to some extremely obscure component that nobody else can relate to. It is simply a single-threaded scheduler with its own synchronization context! Let me know if I'm overlooking something that already exists in the base libraries of .Net.

Thanks again.

ramya-rao-a commented 3 years ago

@JoshLove-msft Anything to add here from the perspective of the newer Azure.Messaging.ServiceBus or our new SDK guidelines?

JoshLove-msft commented 3 years ago

The new library does not support customizing how tasks are scheduled. It uses the ThreadPool threads just like Microsoft.Azure.ServiceBus.

JoshLove-msft commented 3 years ago

/cc @tg-msft @pakrym are there any SDK-wide guidelines around task scheduling implementations?

JoshLove-msft commented 3 years ago

To clarify, the callback is executed inline in the ServiceBusProcessor, i.e. not using Task.Run. You are free to use whatever scheduling mechanism you prefer from within your callback to schedule work. @dbeavon does this address your concern?

ghost commented 3 years ago

Hi, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!