sumerc / yappi

Yet Another Python Profiler, but this time multithreading, asyncio and gevent aware.
MIT License
1.44k stars 72 forks source link

Fastapi support #98

Closed fansonfong closed 1 year ago

fansonfong commented 2 years ago

I am new to yappi and fastapi, I want to use yappi to profile my fastapi app with uvicorn as WSGI server, I googled it for a while and also checked yappi official site, but failed to find a direct solution for this. Does anyone share to me how to use yappi in fastapi? Super thx.

sumerc commented 2 years ago

Hi.

This is the issue we worked with few FastAPI devs on the subject: https://github.com/tiangolo/fastapi/issues/701 and the outcome of that issue was a simple middleware that you can use in your FastAPI(in fact any ASGI) application:

https://github.com/sm-Fifteen/asgi-server-timing-middleware

fansonfong commented 2 years ago

Hi.

This is the issue we worked with few FastAPI devs on the subject: tiangolo/fastapi#701 and the outcome of that issue was a simple middleware that you can use in your FastAPI(in fact any ASGI) application:

https://github.com/sm-Fifteen/asgi-server-timing-middleware

Hi @sumerc , thank you for your answer, however, I cannot install yappi in my python project with poetry as dependency management tool. I tried with different yappi version, but still failed. Here is the error stack trace in terminal:

source /Users/frank/Desktop/my-python-project/.venv/bin/activate
/Users/frank/Library/Python/3.8/bin/poetry add yappi
Using version ^1.3.3 for yappi

Updating dependencies
Resolving dependencies... (0.4s)

Package operations: 1 install, 0 updates, 0 removals

  • Installing yappi (1.3.3): Failed

  EnvCommandError

  Command ['/Users/frank/Desktop/my-python-project/.venv/bin/pip', 'install', '--no-deps', 'file:///Users/frank/Library/Caches/pypoetry/artifacts/f0/33/32/1e69af583ac557e41859c1bc9b67bc2c90376b7e9be988885c2f993cd8/yappi-1.3.3.tar.gz'] errored with the following return code 1, and output: 
  Processing /Users/frank/Library/Caches/pypoetry/artifacts/f0/33/32/1e69af583ac557e41859c1bc9b67bc2c90376b7e9be988885c2f993cd8/yappi-1.3.3.tar.gz
    Preparing metadata (setup.py): started
    Preparing metadata (setup.py): finished with status 'done'
  Building wheels for collected packages: yappi
    Building wheel for yappi (setup.py): started
    Building wheel for yappi (setup.py): finished with status 'error'
    error: subprocess-exited-with-error

    × python setup.py bdist_wheel did not run successfully.
    │ exit code: 1
    ╰─> [71 lines of output]
        running bdist_wheel
        running build
        running build_py
        creating build
        creating build/lib.macosx-11-x86_64-cpython-39
        copying yappi/yappi.py -> build/lib.macosx-11-x86_64-cpython-39
        running build_ext
        building '_yappi' extension
        creating build/temp.macosx-11-x86_64-cpython-39
        creating build/temp.macosx-11-x86_64-cpython-39/yappi
        clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/Users/frank/Desktop/my-python-project/.venv/include -I/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c yappi/_yappi.c -o build/temp.macosx-11-x86_64-cpython-39/yappi/_yappi.o
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/private/var/folders/tx/yx40gd_107ndmpszb91zqbvh0000gn/T/pip-req-build-qv5jvq9w/setup.py", line 65, in <module>
            setup(
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
            return distutils.core.setup(**attrs)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
            return run_commands(dist)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
            dist.run_commands()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
            self.run_command(cmd)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
            super().run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
            cmd_obj.run()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
            self.run_command('build')
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
            super().run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
            cmd_obj.run()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 136, in run
            self.run_command(cmd_name)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
            super().run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
            cmd_obj.run()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
            _build_ext.run(self)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
            self.build_extensions()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
            self._build_extensions_serial()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
            self.build_extension(ext)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
            _build_ext.build_extension(self, ext)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 528, in build_extension
            objects = self.compiler.compile(sources,
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 574, in compile
            self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 173, in _compile
            self.spawn(compiler_so + cc_args + [src, '-o', obj] +
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 917, in spawn
            spawn(cmd, dry_run=self.dry_run, **kwargs)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/spawn.py", line 56, in spawn
            proc = subprocess.Popen(cmd, env=env)
          File "/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 947, in __init__
            self._execute_child(args, executable, preexec_fn, close_fds,
          File "/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1739, in _execute_child
            env_list.append(k + b'=' + os.fsencode(v))
          File "/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 810, in fsencode
            filename = fspath(filename)  # Does type-checking of `filename`.
        TypeError: expected str, bytes or os.PathLike object, not int
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for yappi
    Running setup.py clean for yappi
  Failed to build yappi
  Installing collected packages: yappi
    Running setup.py install for yappi: started
    Running setup.py install for yappi: finished with status 'error'
    error: subprocess-exited-with-error

    × Running setup.py install for yappi did not run successfully.
    │ exit code: 1
    ╰─> [75 lines of output]
        running install
        /Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        running build
        running build_py
        creating build
        creating build/lib.macosx-11-x86_64-cpython-39
        copying yappi/yappi.py -> build/lib.macosx-11-x86_64-cpython-39
        running build_ext
        building '_yappi' extension
        creating build/temp.macosx-11-x86_64-cpython-39
        creating build/temp.macosx-11-x86_64-cpython-39/yappi
        clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/Users/frank/Desktop/my-python-project/.venv/include -I/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c yappi/_yappi.c -o build/temp.macosx-11-x86_64-cpython-39/yappi/_yappi.o
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/private/var/folders/tx/yx40gd_107ndmpszb91zqbvh0000gn/T/pip-req-build-qv5jvq9w/setup.py", line 65, in <module>
            setup(
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
            return distutils.core.setup(**attrs)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
            return run_commands(dist)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
            dist.run_commands()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
            self.run_command(cmd)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
            super().run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
            cmd_obj.run()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/command/install.py", line 68, in run
            return orig.install.run(self)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 670, in run
            self.run_command('build')
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
            super().run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
            cmd_obj.run()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 136, in run
            self.run_command(cmd_name)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
            super().run_command(command)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
            cmd_obj.run()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
            _build_ext.run(self)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
            self.build_extensions()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 448, in build_extensions
            self._build_extensions_serial()
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 473, in _build_extensions_serial
            self.build_extension(ext)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 202, in build_extension
            _build_ext.build_extension(self, ext)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 528, in build_extension
            objects = self.compiler.compile(sources,
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 574, in compile
            self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 173, in _compile
            self.spawn(compiler_so + cc_args + [src, '-o', obj] +
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 917, in spawn
            spawn(cmd, dry_run=self.dry_run, **kwargs)
          File "/Users/frank/Desktop/my-python-project/.venv/lib/python3.9/site-packages/setuptools/_distutils/spawn.py", line 56, in spawn
            proc = subprocess.Popen(cmd, env=env)
          File "/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 947, in __init__
            self._execute_child(args, executable, preexec_fn, close_fds,
          File "/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1739, in _execute_child
            env_list.append(k + b'=' + os.fsencode(v))
          File "/usr/local/Cellar/python@3.9/3.9.1_7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 810, in fsencode
            filename = fspath(filename)  # Does type-checking of `filename`.
        TypeError: expected str, bytes or os.PathLike object, not int
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
  error: legacy-install-failure

  × Encountered error while trying to install package.
  ╰─> yappi

  note: This is an issue with the package mentioned above, not pip.
  hint: See above for output from the failure.

  at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/utils/env.py:1195 in _run
      1191│                 output = subprocess.check_output(
      1192│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1193│                 )
      1194│         except CalledProcessError as e:
    → 1195│             raise EnvCommandError(e, input=input_)
      1196│ 
      1197│         return decode(output)
      1198│ 
      1199│     def execute(self, bin, *args, **kwargs):

Failed to add packages, reverting the pyproject.toml file to its original content.
sumerc commented 1 year ago

We have recently released a new version of yappi (1.3.6) that has wheels compiled for most of the platforms. I would suggest using that version.

I am closing the issue as I have suggested a way to use a middleware for FastAPI.