Open jack-zhang-ai opened 1 month ago
wow - thats really interesting. I just tested this out, and can replicate this on my dev machine. It does seem like we can profile the python-build-standalone
interpreter without an issue - but after going import shapely.geos
we can't.
Taking a quick look at this, it seems like the import shapely.geos
adds a new entry in the /proc/PID/maps
virtual memory maps pointing to a new executable section to libpython.so. Like before importing shapely.geos we have only
[2024-11-01T21:33:43.828444634Z DEBUG py_spy::python_spy] map: 00007f83dc641000-00007f83dd31e000 r-x /home/ben/code/python-standalone/python/lib/libpython3.9.so.1.0
but after importing we have
[2024-11-01T21:34:09.723721219Z DEBUG py_spy::python_spy] map: 00007f83dc4c3000-00007f83dc4c4000 r-x /home/ben/code/python-standalone/python/lib/libpython3.9.so.1.0
[2024-11-01T21:34:09.723760971Z DEBUG py_spy::python_spy] map: 00007f83dc641000-00007f83dd31e000 r-x /home/ben/code/python-standalone/python/lib/libpython3.9.so.1.0
The issue seems to be that py-spy ends up trying to load data from the first smaller section that shapely.geos has imported - and should be using the second one.
We are hitting the following issue when trying to py-spy while using a Python distribution from python-build-standalone and importing shapely.geos from shapely==1.8.5.
A bit of digging makes me suspect that its some how related to shapely <2.0 dynamically linking to GEOS. Notably, shapely >2 does work (which we're working on upgrading shapely to avoid this as well).
Other details:
ubuntu:24.04
containerimport shapely.geos
print(os.getpid()) while True: time.sleep(1)