search

DataDog / saluki

An experimental toolkit for building telemetry data planes in Rust.
Apache License 2.0
12 stars 2 forks source link

Consider how to load and share the root certificate store across all HTTP client usages. #182

Closed tobz closed 4 weeks ago

tobz commented 1 month ago

While examining component bounds vs actual usage, I noticed that one component was managing to use ~115KiB when it should have been using <10KiB. Doing some investigation led me to discover that this component just happened to be the first one to create an HttpClient, which meant it was responsible for (among other things) lazily loading the root certificate store in the process.

The root certificate store is worth about ~85KiB on my local development machine (Ubuntu 23.10) which isn't necessarily surprising, but certainly blows the budget of the component in the context of its bounds.

Looking deeper into rustls, we can pass it an Arc<T>-wrapped version of the root certificate store that should be used, which means we could conceptually initialize it early on and attribute that memory usage to the root allocation group instead, which would make things look nicer.

The biggest issue is just figuring out how/where to store the shared value for each access. Perhaps we could/should expose some free functions in saluki_io to initialize a OnceCell, which would then be called by saluki_app::tls::initialize_tls.. and then from that point, we'd just use that when creating HTTP clients.

:shrug:

tobz commented 4 weeks ago

This is going to be solved by #205 which has us loading the platform's native root certificate store and making it reusable/sharable by default when a client TLS configuration is generated.