neondatabase / neon

Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
https://neon.tech
Apache License 2.0
15.06k stars 439 forks source link

Support dynamic tracing per Tenant/Timeline/... #3547

Open shanyp opened 1 year ago

shanyp commented 1 year ago

Original comment from @kelvich:

It would also be super useful to have:

per-module tags on logs and the ability to enforce per-module log level in a runtime, e.g. through http api. ability to enforce all logs for a specific tenant in runtime depending on complexity that could be done now or later

### Tasks
- [ ] https://github.com/neondatabase/neon/pull/3374
SomeoneToIgnore commented 1 year ago

Status:

The PR misses one crucial thing that I'd love to do somehow: a benchmark comparison against main to see if the performance could've degraded somehow. That could be due to the nature of the chage in the PR into tracing: to make tracing logger to react to filter changes dynamically, it has to make more checks after the patch everywhere where spans are used.

For such a global change, it would be good to have a benchmark with multiple tenant and timelines involved, no extra log filtering features enabled, comparing two versions of tracing. I am not aware of any way to interpret current benchmark results, neither of the way to display them on the charts.