parca-dev / parca-agent

eBPF based always-on profiler auto-discovering targets in Kubernetes and systemd, zero code changes or restarts needed!
https://parca.dev/
Apache License 2.0
509 stars 67 forks source link
bpf c continuous-profiling cpp ebpf ebpf-programs go golang hacktoberfest kubernetes libbpf linux observability performance pprof profiling python ruby rust systemd

Apache 2 License Build Container parca-agent

Parca Agent

Parca Agent is an always-on sampling profiler that uses eBPF to capture raw profiling data with very low overhead. It observes user-space and kernel-space stacktraces 19 times per second and builds pprof formatted profiles from the extracted data. Read more details in the design documentation.

The collected data can be viewed locally via HTTP endpoints and then be configured to be sent to a Parca server to be queried and analyzed over time.

Requirements

Quickstart

See the Kubernetes Getting Started.

Language Support

Parca Agent is continuously enhancing its support for multiple languages. Incomplete list of languages we currently support:

Please check our docs for further information.

[!NOTE] Further language support is coming in the upcoming weeks and months.

Supported Profiles

Types of profiles that are available:

[!NOTE] Please check our docs if your language is supported.

The following types of profiles require explicit instrumentation:

Debugging

Web UI

The HTTP endpoints can be used to inspect the active profilers, by visiting port 7071 of the process (the host-port that the agent binds to can be configured using the --http-address flag).

On a minikube cluster that might look like the following:

Active Profilers

And by clicking "Show Profile" in one of the rows, the currently collected profile will be rendered once the collection finishes (this can take up to 10 seconds).

Profile View

A raw profile can also be downloaded here by clicking "Download Pprof". Note that in the case of native stack traces such as produced from compiled language like C, C++, Go, Rust, etc. are not symbolized and if this pprof profile is analyzed using the standard pprof tooling the symbols will need to be available to the tooling.

Logging

To debug potential errors, enable debug logging using --log-level=debug.

Configuration

Flags:

[embedmd]:# (dist/help.txt) ```txt Usage: parca-agent Flags: -h, --help Show context-sensitive help. --log-level="info" Log level. --log-format="logfmt" Configure if structured logging as JSON or as logfmt --http-address="127.0.0.1:7071" Address to bind HTTP server to. --version Show application version. --node="hostname" The name of the node that the process is running on. If on Kubernetes, this must match the Kubernetes node name. --config-path="" Path to config file. --memlock-rlimit=0 The value for the maximum number of bytes of memory that may be locked into RAM. It is used to ensure the agent can lock memory for eBPF maps. 0 means no limit. --mutex-profile-fraction=0 Fraction of mutex profile samples to collect. --block-profile-rate=0 Sample rate for block profile. --profiling-duration=10s The agent profiling duration to use. Leave this empty to use the defaults. --profiling-cpu-sampling-frequency=19 The frequency at which profiling data is collected, e.g., 19 samples per second. --profiling-perf-event-buffer-poll-interval=250ms The interval at which the perf event buffer is polled for new events. --profiling-perf-event-buffer-processing-interval=100ms The interval at which the perf event buffer is processed. --profiling-perf-event-buffer-worker-count=4 The number of workers that process the perf event buffer. --metadata-external-labels=KEY=VALUE;... Label(s) to attach to all profiles. --metadata-container-runtime-socket-path=STRING The filesystem path to the container runtimes socket. Leave this empty to use the defaults. --metadata-disable-caching Disable caching of metadata. --local-store-directory=STRING The local directory to store the profiling data. --remote-store-address=STRING gRPC address to send profiles and symbols to. --remote-store-bearer-token=STRING Bearer token to authenticate with store ($PARCA_BEARER_TOKEN). --remote-store-bearer-token-file=STRING File to read bearer token from to authenticate with store. --remote-store-insecure Send gRPC requests via plaintext instead of TLS. --remote-store-insecure-skip-verify Skip TLS certificate verification. --remote-store-batch-write-interval=10s Interval between batch remote client writes. Leave this empty to use the default value of 10s. --remote-store-rpc-logging-enable Enable gRPC logging. --remote-store-rpc-unary-timeout=5m Maximum timeout window for unary gRPC requests including retries. --debuginfo-directories=/usr/lib/debug,... Ordered list of local directories to search for debuginfo files. --debuginfo-temp-dir="/tmp" The local directory path to store the interim debuginfo files. --debuginfo-strip Only upload information needed for symbolization. If false the exact binary the agent sees will be uploaded unmodified. --debuginfo-compress Compress debuginfo files' DWARF sections before uploading. --debuginfo-upload-disable Disable debuginfo collection and upload. --debuginfo-upload-max-parallel=25 The maximum number of debuginfo upload requests to make in parallel. --debuginfo-upload-timeout-duration=2m The timeout duration to cancel upload requests. --debuginfo-upload-cache-duration=5m The duration to cache debuginfo upload responses for. --debuginfo-disable-caching Disable caching of debuginfo. --symbolizer-jit-disable Disable JIT symbolization. --otlp-address=STRING The endpoint to send OTLP traces to. --otlp-exporter="grpc" The OTLP exporter to use. --object-file-pool-eviction-policy="lru" The eviction policy to use for the object file pool. --object-file-pool-size=100 The maximum number of object files to keep in the pool. This is used to avoid re-reading object files from disk. It keeps FDs open, so it should be kept in sync with ulimits. 0 means no limit. --dwarf-unwinding-disable Do not unwind using .eh_frame information. --dwarf-unwinding-mixed Unwind using .eh_frame information and frame pointers. --python-unwinding-disable Disable Python unwinder. --ruby-unwinding-disable Disable Ruby unwinder. --analytics-opt-out Opt out of sending anonymous usage statistics. --telemetry-disable-panic-reporting --telemetry-stderr-buffer-size-kb=4096 --bpf-verbose-logging Enable verbose BPF logging. --bpf-events-buffer-size=8192 Size in pages of the events buffer. --verbose-bpf-logging [deprecated] Use --bpf-verbose-logging. Enable verbose BPF logging. ```

Security

Parca Agent is required to be running as root user (or CAP_SYS_ADMIN). Various security precautions have been taken to protect users running Parca Agent. See details in Security Considerations.

To report a security vulnerability, see this guide.

Contributing

Check out our Contributing Guide to get started!

License

User-space code: Apache 2

Kernel-space code (eBPF profilers): GNU General Public License, version 2

Credits

Thanks to: