unwinding/dwarf: arm64 support

Context

While we currently build arm64 containers for the Agent, we don't truly support BPF-based profiling anymore. This is due to a complete rework of the core BPF bits to unwind through code without frame pointers that focused on supporting x86_64.

We want to change this, as there's an ever-growing amount of developers using this architecture locally, see Apple Silicon, as well as in production (Graviton et al).

Roadmap

Enabling work

[x] https://github.com/parca-dev/parca-agent/commit/9f6546ae1887151fd9f4b8839d93982d7a8fb1fe Per-architecture vmlinux.h. As of this time, we only have the one for x86_64, we need to have one per architecture, and choose the right one at build time. This is needed CO-RE and BTF support ;
[x] https://github.com/parca-dev/parca-agent/commit/abdaf108a7c343b43d57909ca6e0356b2ade8cc9 Build the agent in arm64 (we only release it now, but we don't build it in normal CI runs AFAIK);
[x] Run CI tests in arm64 to ensure further changes (build tags, unwinder changes, etc) don't break our code under this architecture;

Adding frame pointer unwinding

[x] https://github.com/parca-dev/parca-agent/commit/9f6546ae1887151fd9f4b8839d93982d7a8fb1fe Frame pointer unwinding is significantly simpler than DWARF-based unwinding. Very small changes are required both in the BPF code and userspace to make it work, so we'll start here, focusing on Go apps;

DWARF-based unwinding (https://github.com/parca-dev/parca-agent/issues/1209)

[x] Ensure our unwind info reader and parser can deal with arm64 binaries (e.g. the info being in the same ELF section, always being present according to its ABI, etc)
[x] Adapt our unwind info parser/interpreter to work with multiple architectures, particularly, arm64;
[x] Adapting the compact unwind row, as well as the unwinder code for arm64's saved return address (link register);

Other things to look out for:

[x] Hardcoded DWARF expressions (plt1 and plt2);
[x] Pointer authentication;
[ ] Split stacks;
[ ] Port JIT test program to arm64;

Testing

[x] The profiler's integration tests must pass;
[x] Checking that things work fine on a VM in an Apple Silicon machine;
[x] Test on Graviton machines;
[x] vmtests for arm64 as a smoke test that we can load the BPF program;
[ ] Given that BPF on arm64 could be less mature, as a stretch goal, we should try
- [ ] multiple distros, as they might have different patches, kernel versions, configs etc;
- [ ] kernel with lockdep enabled

parca-dev / parca-agent

unwinding/dwarf: arm64 support #1797