Closed ray-ray closed 1 year ago
Same issue here: https://github.com/GoogleCloudPlatform/microservices-demo/pull/1318 with python:3.11.0
but not with python:3.10.8
.
In file included from googlecloudprofiler/src/profiler.h:26,
from googlecloudprofiler/src/_profiler.cc:18:
googlecloudprofiler/src/stacktraces.h: In member function ‘void AsyncSafeTraceMultiset::Reset()’:
googlecloudprofiler/src/stacktraces.h:72:52: warning: ‘void* memset(void*, int, size_t)’ clearing an object of type ‘struct AsyncSafeTraceMultiset::TraceData’ with no trivial copy-assignment; use value-initialization instead [-Wclass-memaccess]
72 | void Reset() { memset(traces_, 0, sizeof(traces_)); }
| ^
googlecloudprofiler/src/stacktraces.h:89:10: note: ‘struct AsyncSafeTraceMultiset::TraceData’ declared here
89 | struct TraceData {
| ^~~~~~~~~
× Encountered error while trying to install package.
╰─> google-cloud-profiler
The actual issue is that the build is broken with 3.11, it is not specific to docker. The breaking change in the Python C API is https://docs.python.org/3/whatsnew/3.11.html#pyframeobject-3-11-hiding. I'm working on a fix
Hey folks. #131 potentially adds support for Python 3.11.
However, I am not particularly confident that this fix is safe/correct. The handler operates possibly without the GIL, as it does not currently modify any of the PyObjects it works with. It gets the thread local ThreadState with PyGILState_GetThisThreadState()
.
PyCodeObject
s may be GC'ed while the handler is running, but we get around that by patching the PyCode deallocator function PyCode_Type.tp_dealloc = &CodeDealloc
The problem is that the getters described in the changelog (PyThreadState_GetFrame()
, PyFrame_GetCode()
, PyFrame_GetBack()
) increment the refcount for those PyObjects which afaict is NOT safe to do without holding the GIL. Trying to acquire the GIL in the handler seems to deadlock some of the internal integration tests; I'm also not confident that this would be async-signal-safe.
Folks are welcome to try the profiler in Python 3.11 from this PR but I am not comfortable merging it as is. If you do try it, please report back on if you encountered any issues. I unfortunately don't have the time to work more on getting this merged right now.
Is there any update on this? 🙏🏻
I took another approach and just merged the PR. Hopefully should see a release next week, but you're welcome to try it from HEAD.
pip install fails using this Dockerfile (from https://cloud.google.com/profiler/docs/profiling-python#using-profiler ):
Here is the build output:
python:3
is equivalent topython:3.11.0
The install does succeed if I instead usepython:3.10.8