[FR] L7 tracing for HTTPs endpoints

dirtyren commented 1 year ago

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description

It would be nice to have L7 tracing for https endpoints

Use case

https://blog.px.dev/ebpf-openssl-tracing/

Related issues

No response

Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

sharang commented 1 year ago

We have supported tracing HTTPS in v6.2 by using eBPF uprobe to obtain the data before encryption. Currently supports Golang applications and applications using the openssl library.

In v6.4, we plan to also trace the SSL/TLS handshake process, so that the time consuming of the handshake phase can be seen in distributed tracing.

In order to enable HTTPS tracing for Golang or openssl, you need to add the following agent configuration:

    ## Regex for Process Name
    ## Note: The name of the process where each feature of ebpf uprobe takes effect,
    ##   which is configured using regular expressions
    #uprobe-process-name-regexs:

      ## Note: Process name to enable Golang-specific symbol table parsing. The default
      ##   value is "", which means that this feature is not enabled for any process.
      ##   This feature acts on Golang processes that have trimmed the standard symbol
      ##   table. For example, the management process of K8s generally belongs to this
      ##   category. When this feature is enabled, for processes with Golang
      ##   version >= 1.13 and < 1.18, when the standard symbol table is missing, the
      ##   Golang-specific symbol table will be parsed to complete uprobe data collection.
      ##   Note that enabling this feature may cause the eBPF initialization process to
      ##   take ten minutes.
      #golang-symbol: ""

      ## Note: The name of the Golang process that enables HTTP2/HTTPS protocol data collection
      ##   and auto-tracing. go auto-tracing also dependent go-tracing-timeout.
      ##   The default value is "", which means it is disabled for all Golang processes.
      #golang: ""

      ## Note: The name of the process that uses the openssl library to enable HTTPS
      ##   protocol data collection. The default is "", which means that it is disabled for
      ##   all processes that use the openssl library.
      #openssl: ""

https://github.com/deepflowio/deepflow/blob/main/server/controller/model/agent_group_config_example.yaml#L1047

sharang commented 10 months ago

In the community edition, we have already supported the tracing capabilities of OpenSSL and Golang TLS, and in v6.4, we expanded the originally supported HTTPS protocol parsing to include all application protocols (gRPC/MySQL/... over TLS, etc.).

dirtyren commented 10 months ago

hey @sharang , I am unable to see any tracings for https endpoints on my K8S cluster using deepflow still. I just wanted to let you know.

deepflowio / deepflow