benfred / py-spy

Sampling profiler for Python programs
MIT License
12.16k stars 401 forks source link

Failed to parse process status when has one more character ')' #566

Open 1C4nfaN opened 1 year ago

1C4nfaN commented 1 year ago

Below are the contents of /proc/{pid}/stat for two processes.

1981866 (python3) S 1981743 1981866 1981743 0 -1 1077936384 34709484 16140 105 0 72770 37613 11 9 20 0 17 0 372814758 4442140672 55046 18446744073709551615 94888328609792 94888328612516 140731140400576 0 0 0 0 16781312 17642 0 0 0 17 79 0 0 16 0 0 94888330710392 94888330711088 94888354955264 140731140409715 140731140409755 140731140409755 140731140415457 0

1984043 (probe(1981866)) S 1981866 1981866 1981743 0 -1 4194368 1324 0 0 0 169 99 0 0 20 0 3 0 372915682 4424044544 33799 18446744073709551615 94888328609792 94888328612516 140731140400576 0 0 0 0 16781312 17642 0 0 0 17 63 0 0 0 0 0 94888330710392 94888330711088 94888354955264 140731140409715 140731140409755 140731140409755 140731140415457 0

When we try to use py-spy dump --pid 1984043, then we get the following error Error: Failed to parse /proc/1984043/stat.

I think the problem should lie in the judgment of this code for the case of '))' (in remoteprocess)

fn get_active_status(stat: &[u8]) -> Option<u8> {
    // find the first ')' character, and return the active status
    // field which comes after it
    let mut iter = stat.iter().skip_while(|x| **x != b')');
    match (iter.next(), iter.next(), iter.next()) {
        (Some(b')'), Some(b' '), ret) => ret.map(|x| *x),
        _ => None
    }
}

I thought I'd make a note of the issue here, though I'm not very familiar with the code that actually executes in process 1984043, I will try to check this and if necessary I'd willing to fix it if I had time.

1C4nfaN commented 1 year ago

It seems that the function doesn't cope with process comms that have ) in them. Not a very common case for Python processes, but I suppose no reason not to support it properly. The logic needs to be "parse from the first ( until the matching )" or maybe just "parse the last )" instead of "the first )".

from @Jongy

Jongy commented 1 year ago

Originally posted: https://github.com/benfred/py-spy/issues/565#issuecomment-1505067511

agausmann commented 4 months ago

I also encountered this. In my case, I think the culprit was PyQt5 and QThreadPool. It looks like this:

$ cat /proc/13793/stat
13793 (Thread (pooled)) S 13689 13689 13688 34828 13689 4194368 64199 549 0 ...

For anyone else who runs into this, it seems to already be fixed in remoteprocess 0.4.12 and py-spy master branch.