search

parca-dev / parca-agent

eBPF based always-on profiler auto-discovering targets in Kubernetes and systemd, zero code changes or restarts needed!
https://parca.dev/
Apache License 2.0
549 stars 68 forks source link

Filter out frames that don't belong to any mapping #380

Open brancz opened 2 years ago

brancz commented 2 years ago

Maybe this is something we should only do when stack unwinding works, but the addresses that are outside of any mapping have no chance of ever being symbolized, so either we set them to <unknown> or we remove them entirely. The only thing I slightly worry about is whether we might need any of that data with interpreted languages (maybe @javierhonduco can weigh in).

Right now it just looks confusing, and is never going to add to a user's understanding.

javierhonduco commented 2 years ago

@brancz I think it makes sense to remove these if the memory is completely unmapped. Semi-related issue: https://github.com/parca-dev/parca-agent/issues/359

kakkoyun commented 2 years ago

I want to make sure we cover all the ground for unmapped memory issues first. Why this actually happens? Stack unwinding an issue we know that causes this, are there any plausible theory what might cause receiving unmapped addresses?

javierhonduco commented 1 year ago

There are two main reasons why this happens right now:

  1. Out-of-date mappings, which we cache as soon as we know about a process. If the process loads a new executable mapping, we might not have it in our cache. This is something we are working on in https://github.com/parca-dev/parca-agent/pull/1507;
  2. The frame pointer unwinder uses a bunch of heuristics to decide when to stop, which might be wrong in some cases, we can easily detect that in the Agent because the address is not within any executable mappings, i.e, the agent failed with a normalization error for address 0x1702484c7480 and pid 70331 sudo cat /proc/70331/maps | grep 170 shows no results