Closed GabrielBianconi closed 6 months ago
Thank you for detailed description. Definitely open to it, PR is welcome!
I'm curious what happens with TLS/DNS on creating a new reqwest
client?
Great, just sent a PR #197.
During initialization, among other things, a reqwest
client seems to initialize its TLS backend ref. In certain scenarios (at least for me: tiny VM with shared core + legacy backend), this seems to take a non-trivial amount of time. I observed ~30ms earlier this week. And with the current async-openai
implementation, it wasn't possible to easily circumvent this dummy initialization.
Thanks!
I see, thank you for sharing!
Initializing a
Client
with a custom HTTP client always requires first creating a uselessreqwest::Client
inClient::with_config
.This adds a bit of useless overhead due to
reqwest
's TLS/DNS functionality, which can be non-trivial for certain low-latency applications.Would you be open to adding an additional method along the lines of?