yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.05k stars 1.08k forks source link

[YSQL] Add distributed traceability framework to link all logs generated in context of the same external client request #14329

Open pkj415 opened 2 years ago

pkj415 commented 2 years ago

Jira Link: DB-3758

Description

Right now, there is no way to establish a 1-1 mapping between log lines in a tserver process’ log and the corresponding log lines in the YSQL logs that resulted in the work on the tserver.

Proposal -

To address this properly, a framework needs to be set up that will allow passage of a unique identifier across all components in the database over RPCs. The unique identifier will be prefixed with all logs/ traces and all the logs/ traces that were emitted in context of the same request from an external client (external to database) will have the same identifier.

The unique identifier will either be supplied by the client external to the database or generated randomly as soon as a new external request is received.

Such a tracing framework should be built on some widely used tracing standards (like Open Tracing) so that existing observability tools that support the standard can be leveraged.Right now, there is no way to establish a 1-1 mapping between log lines in a tserver process’ log and the corresponding log lines in the YSQL logs that resulted in the work on the tserver.

SteveXiSong commented 2 years ago

CC @sushantrmishra @anmalysh-yb @nbhatia-yb