When attaching with --pid=, shared libraries are not covered

VincentGibert commented 9 years ago

Hi (again),

When I use kcov with the --pid flag, I don't manage to get coverage results.

Environment

Operating system: CentOS 6.6 x86_64
kcov version: git snapshot from revision 157d013 which fixes #104.

Processus executed: /usr/sbin/snmpd from net-snmp-5.5 compiled using -g -O0 (snmpd binary is not stripped, debug symbols are not in a separate build id).

Steps to reproduce

Launch the snmpd daemon

$ /usr/sbin/snmpd

Attach kcov to the running snmpd instance

$ ps -ef | grep snmpd
root      8307      1  0 16:03 ?        00:00:00 /usr/sbin/snmpd
$ kcov -p 8307 /tmp/kcov_results /usr/sbin/snmpd &

Make sure snmpd does something and kill it

$ snmpwalk -v 2c -c public localhost # make sure snmpd does something
$ kill 8307

Coverage results generated by kcov are void

$ ls /tmp/kcov_results/snmpd
amber.png  bcov.css  cobertura.xml  coverage.db  data  glass.png  index.html  index.json  metadata  summary.db

Additional information

If the snmp daemon is executed directly by kcov, coverage results are correctly generated.

$ kcov /tmp/kcov_results /usr/sbin/snmpd &
$ pkill snmpd
kcov: Process exited with signal 15 (SIGTERM) at 0x7ffff77d4d4d
$ ls /tmp/kcov_results/snmpd/ # 628 results
agent_handler.c.df3bdac7.html agent_handler.c.df3bdac7.json agent_index.c.4976cbcb.html agent_index.c.4976cbcb.json [...] snmp.c.1eaa0f6.html snmp.c.1eaa0f6.json ...

This issue also occurs when snmpd is launched using -f flag which prevents snmpd from forking.

When running kcov with --debug=15, the following message appears:

$ kcov -p 8283 --debug=15  /tmp/kcov_results /usr/sbin/snmpd
PT stopped PID 8283 0x0000137f
PT BP at 0x7f15c61e8432:-1 for 8283
Can't find breakpoint at 0x7f15c61e8432
PT continuing 8283 with signal 0

What does it mean ? Maybe some debug information are missing on my system ?

SimonKagstrom commented 9 years ago

Sorry, I've completely missed this bug report.

It might have something to do with shared library support I think, which perhaps isn't working well together with the --pid= option. Shared libraries are handled via a LD_PRELOAD override, but for an already started process, the LD_PRELOAD wasn't set at startup time. Therefore kcov doesn't detect the loaded libraries when it starts.

You should still get hits from snmpd itself, but not from the other libraries loaded by it.

VincentGibert commented 9 years ago

It makes sense, I will try with a static binary to confirm your analysis.

VincentGibert commented 9 years ago

I just tested with a static version of busybox and it worked well so it seems you guessed right.

SimonKagstrom commented 9 years ago

Great, thanks for testing!

I think I'll resolve this issue by adding a few lines to the documentation - I don't think it will be very easy to fix the core issue.

VincentGibert commented 9 years ago

You may probably retrieve the list of loaded shared libraries by the process through /proc//maps (sadly it is Linux specific) but I don't know if it is sufficient to trace them.

SimonKagstrom commented 9 years ago

Yes, maybe that's a possibility. I'll keep the bug report open for further investigation.

SimonKagstrom commented 8 years ago

I took a look at this bug again, and I believe it should be doable to parse /proc//maps for the shared library-when-tracing-via-pid case.

It will still not cover all cases since it will be difficult to detect dlopen():ing shared libraries this way (which is done via the LD_PRELOAD stuff above as well).

Anyway, it will be Linux-only, but the shared library handling depends on dl_iterate_phdr() as it is today, so it's already Linux-specific.

SimonKagstrom / kcov