Open dongzeli95 opened 4 months ago
So upon load testing, I found that if I have 30 concurrent streaming recognize sessions, it would create problem where all threads stuck. I have tried to use both gevent greenlet and native thread pool, nothing helps with this limit.
On the other hand, concurrency under 25 seems to work fine. I've also seen this post arguing about grpc max concurrency settings. I assume we are using grpc underneath this speech recognition package. Is there a way we can increase concurrency on speech client?
Could someone help take a look? Thank you so much!
Hi @dongzeli95 ,
Thanks for reporting this issue. This is potentially related to https://github.com/grpc/grpc/issues/36265, https://github.com/googleapis/python-bigtable/issues/949 and https://github.com/googleapis/google-cloud-python/issues/12423. To confirm, can you try downgrading to grpcio==1.58.0
?
@parthea Just confirmed that changing dependency onto grpcio==1.58.0
didn't help with this issue. Hope that helps.
This is my code for running each recognize task in a separate gevent greenlet and this causes the greenlet to stuck:
I've also included patch for my gevent application:
So after the above greenlet stuck, my whole application will basically hang.
Note that if I do the stream recognition synchronously without using gevent greenlet, it works fine. However I still prefer to use it in separate thread to improve on latency. I wonder if this is also another grpc incompatibility issue with gevent.