search

TranslatorSRI / PerformanceTestRunner

Run performance tests against the Translator system
MIT License
0 stars 0 forks source link

Opinion: Use Asyncio instead of ThreadPoolExecutor #4

Open maximusunc opened 3 months ago

maximusunc commented 3 months ago

I'm not sure a ThreadPoolExecutor is necessarily the right tool for what we want to do here. Each query doesn't really need its own thread as we aren't doing anything super CPU intensive, and we will arguably want to send more than 30 or so queries at a time. I find this answer to be very informative about the differences: https://stackoverflow.com/a/61360215

I would like to advocate for using asyncio as I think it's more suited for this task, as well as being cleaner in code (no literal parsing of the html response)

ShervinAbd92 commented 3 months ago

switched to asyncio.

maximusunc commented 3 months ago

It looks like the stress_utilities block still uses the ThreadPoolExecutor

maximusunc commented 3 months ago

Also, I think aiohttp is not needed. httpx has plenty of exception types you can use: https://www.python-httpx.org/exceptions/