Each "async" client method consumes a thread

[ccoffing]

With the C++ bindings, the Async*() methods impose no threading model on the client. I can start an async method, then separately on my own thread(s) call cq->Next() and dispatch the completions however I wish. I can service many simultaneous client requests on few threads this way. This is critical for constrained environments.

With grpc-c, the *async() functions behave differently. "async" seems to mean "answered asynchronously to the caller". But each async() call consumes a thread, since internally, they call grpc_completion_queue_next(), which blocks until that particular tag completes. So if I start another client, it will see that the thread pool is full (due to grpc_completion_queue_next blocking each thread) and so will spin up another thread, which itself will block on grpc_completion_queue_next.

I'm curious if this is by design? I would like an option to opt out of the thread pool, and call ..._next() myself.

I see the (unfinished) references to coroutines, and while I have used coroutines before and appreciate them, it still imposes a threading model on the caller.

Do you have plans for allowing the caller to manage the completion queues themselves? I might be able to help, but first want to understand your plans.

Thanks!

[ajhai]

_async() in grpc-c is different from Async() from C++ bindings by design. We didn't want to ask user to deal with completion_queue and other gRPC internals. You are right that we unnecessarily block a thread when doing grpc_completion_queue_next(). Right now we do not have any plans to make changes to allow user to opt completely out of thread pool.

However, there is a plan to fix thread blocking behavior where we want to use just one thread to check for pending tags on all available completion queues (with a timeout). This is how grpc-c works with event based libraries that we use here internally (you can see gc_handle_server_event_internal() in grpc-c get called with different timeouts).

Instead of blocking on grpc_completion_queue_next(), we call it with a timeout. If timed out, we can move this completion queue to a list that gets iterated for new events in a loop in a separate thread. I will try to get this done before moving to beta. Does something like this work for you?

[ccoffing]

For the time being, I am using the C++ bindings in my otherwise-C project (for various reasons: maturity, threading issues, proto generation consistency, and familiarity). So consider me a no-pressure observer on this issue.

My environment is POSIX-ish, with opaque thread pools, and direct usage of pthreads is frowned upon but possible. In my particular usage, my threads can technically be blocked but it will affect scalability. Also, I need to receive across an arbitrary number of cores for scalability.

In my current design, each thread in the thread pool monitors a completion queue. It seems that the point of friction with adopting grpc-c is, who creates/owns the threads? If grpc-c ultimately creates and manages the threads, then that is the choke point of scalability. In my case, it's the user of grpc-c who knows how many cores they can scale across.

[cgrozev]

Hi @ajhai. Have you made any progress on the timeout implementation at grpc_completion_queue_next()? regards