benfred / py-spy

Sampling profiler for Python programs
MIT License
12.16k stars 401 forks source link

[Windows] - Error: failed to find a python interpreter in the .data section #564

Open minerharry opened 1 year ago

minerharry commented 1 year ago

Ran into an error when trying to use a process monitor relying on py-spy as a dependency (ray dashboard) and I was running into an issue ("Invalid access to memory location. (os error 998)"). To debug, I wrote a short test script and used py-spy myself from the terminal on it, but it consistently raised: "Error: failed to find a python interpreter in the .data section". Originally found on a windows 11 machine, but replicated on windows 10. Version specs:

Machine 1: Windows 11 py-spy version 0.3.14 Python version 3.10.9

Machine 2: Windows 10 py-spy version 0.3.14 Python version 3.10.4

On both machines, the following commands return the error: py-spy top -- python pyspy_test.py py-spy top -p [pid from os.getpid] py-spy top -n -p [pid from os.getpid]

I also wanted to try a more canonical test (as it seemed few others had this issue), so I found what seemed to be the unit testing with cargo test --release within a clone of the repo. Interestingly, the two machines do give different test results, though both fail: Machine 1: machine1testout.txt

Machine 2: machine2testout.txt

and the test I was using to debug it: (the random choice was to test stack trace stuff but I ran into this bug instead)

import random
import os

def f1(n):
    result = [];
    for k in range(n):
        for j in range(k):
            result.append(k**j);
    return result;

def f2(n):
    result = [];
    for k in range(n):
        for j in range(k):
            result.append(k**j);
    return result;

def f3(n):
    result = [];
    for k in range(n):
        for j in range(k):
            result.append(k**j);
    return result;

def f4(n):
    result = [];
    for k in range(n):
        for j in range(k):
            result.append(k**j);
    return result;

def f5(n):
    result = [];
    for k in range(n):
        for j in range(k):
            result.append(k**j);
    return result;
print(os.getpid());
while True:
    n = random.randrange(1,100);
    c = random.choice([f1,f2,f3,f4,f5]);
    result = c(n);
Prethea-Phoenixia commented 1 year ago

May be an issue with Windows and Python 3.10? I was able to reproduce your situation with a Windows 10 desktop, Python 3.10.10 and py-spy 0.3.14, but the older Python 3.8.16 on my Windows 7 laptop with the same version of py-spy runs fine.

minerharry commented 1 year ago

This is very strange. I've been testing it on different versions with conda, at it looks like it works fine on all of the conda versions of python, at least down to 3.7 where I stopped testing. This includes 3.10.9. However, running my base python (3.10.9) still breaks. Very frustrating! I guess I'll just be using conda for now.