joerick / pyinstrument

🚴 Call stack profiler for Python. Shows you why your code is slow!
https://pyinstrument.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
6.57k stars 231 forks source link

Pyinstrument ProfilerState object is not callable #300

Closed Partho closed 5 months ago

Partho commented 6 months ago

I am working on using PyInstrument to profile FastAPI requests. When I am trying to make this post call:

curl -X 'POST' \
  'http://0.0.0.0:8000/api/model/lumalee/decisions/result/file/?shap_values=0&libsvm=0' \
  -H 'accept: application/json' \
  -H 'Content-Type: multipart/form-data' \
  -F 'file=@clean_sample'

The task is getting stuck, and when I refresh the web page, I am getting this error:

 asctime=2024-04-16 15:11:20,897, level=ERROR [uvicorn.error] msg=Exception in ASGI application
 Traceback (most recent call last):
   File "/usr/local/lib/python3.9/dist-packages/starlette/middleware/base.py", line 108, in __call__
     response = await self.dispatch_func(request, call_next)
   File "/home/dsci/classifier-api/app/core/middleware.py", line 25, in profile_request
     await call_next(request)
   File "src/memray/_memray.pyx", line 712, in memray._memray.Tracker.__exit__
   File "/usr/lib/python3.9/threading.py", line 58, in setprofile
     def setprofile(func):
 TypeError: 'pyinstrument.stat_profile.ProfilerState' object is not callable

Basically, the error is happening here:

def register_profiler(app: FastAPI):
    @app.middleware("http")
    async def profile_request(request: Request, call_next):
        """
            Profile the CPU and memory of the current request
        """
        if config.PROFILING_ENABLED is True:
            memray_file_path = memray.FileDestination('/tmp/output.bin', overwrite=True)
            with memray.Tracker(destination=memray_file_path, trace_python_allocators=True):
                await call_next(request) <----------------------------------------HERE THE ERROR OCCURS
            subprocess.run(
                ["memray", "summary", "/tmp/output.bin"]
            )

            profiler = Profiler(interval=0.001, async_mode="enabled")
            profiler.start()
            response = await call_next(request)
            profiler.stop()
            profiler.print(show_all=True)
            return response
        return await call_next(request)

Any idea why I am getting not callable error? Due to this error, we won't be able to profile post calls in FastAPI which is a roadblock for us. Please help in resolving this issue. Thanks.

joerick commented 5 months ago

I see you are using memray. It's possible that there's a bug in memray's profiler forwarding mechanism. Or since memray is a profiler too, it might just not be possible to run both at the same time.

Might be worth asking the memray team as from reading the code, it looks like they're trying to restore the previous profiling state and that's what is causing this error.