Wrapper scripts retain pre-exec procname

[igorwwwwwwwwwwwwwwwwwwww]

TLDR

process-exporter tracks the procname for a process the first time it sees it. If that process then performs an execve we do not reconsider it. This leads to processes being ignored even if they would later become matching.

Background

I ran into this while attempting to add process-exporter as a sidecar to the bitnami/redis helm chart (downstream issue).

That chart has a start-node.sh wrapper script that runs some checks before eventually performing an exec redis-server.

Problem

Running process-exporter -procnames redis-server did not match anything. By changing this to process-exporter -procnames start-node.sh I was able to get it working. But now the group name is start-node.sh, which is quite misleading and may also be susceptible to race conditions.

Potential workarounds and solutions

Define regex that matches both start-node.sh and redis-server

Something like this may be the most viable workaround given the current implementation:

process_names:
  - name: redis-server
    comm:
    - start-node.sh
    - redis-server

I still need to verify that this works.

Delay start of process-exporter

A hack I thought about was delaying process-exporter. However, because the docker image is built FROM scratch it does not include a shell.

This makes it impossible to do something like:

sh -c 'sleep 10 && exec /bin/process-exporter'

It could be added as a feature to the go binary, but it's extremely hacky to begin with.

Periodically re-evaluate all processes

If process-exporter were to periodically re-evaluate whether processes (both matched and unmatched) actually should match, it could better respond to the presence of execve, and perhaps change its mind after an initial bad impression.

This seems like the most "correct" solution. However, I don't know how easy this would be to do, and how expensive it would be.

[flixr]

There is already the -recheck flag that means it will always check this again. See also #23 for an option to only recheck for a certain time...

[igorwwwwwwwwwwwwwwwwwwww]

@flixr Oh amazing, -recheck is exactly what I wanted. Thanks!

Closing. :)