search

jordaneremieff / mangum

AWS Lambda support for ASGI applications
https://mangum.io/
MIT License
1.67k stars 119 forks source link

Magnum have huge list of dependencies, many have dubious utility. #264

Closed Fak3 closed 2 years ago

Fak3 commented 2 years ago

Currently magnum installs over 150 pacakages along with it: pdm list --graph

I wish there was a slimmer version, that only installs core depndencies. Currently some deps like yappi and netifaces does not provide binary wheels, which makes magnum installation fail in case of missing python headers:

Preparing isolated env for PEP 517 build...
Building wheel for https://files.pythonhosted.org/packages/9c/bb/47be36b473e56360d3012bf1b6e405785e42d1f2e91da715964c1a705937/yappi-1.3.5.tar.gz#sha256=f54c25f04aa7c613633b529bffd14e0699a4363f414dc9c065616fd52064a49b (from https://pypi.org/simple/yappi/)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting setuptools>=40.8.0
  Using cached setuptools-62.6.0-py3-none-any.whl (1.2 MB)
Installing collected packages: wheel, setuptools
Successfully installed setuptools-62.6.0 wheel-0.37.1
/tmp/timer_create3280tt_g.c: In function ‘main’:
/tmp/timer_create3280tt_g.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
    2 |     timer_create();
      |     ^~~~~~~~~~~~
running egg_info
writing yappi/yappi.egg-info/PKG-INFO
writing dependency_links to yappi/yappi.egg-info/dependency_links.txt
writing entry points to yappi/yappi.egg-info/entry_points.txt
writing requirements to yappi/yappi.egg-info/requires.txt
writing top-level names to yappi/yappi.egg-info/top_level.txt
reading manifest file 'yappi/yappi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'yappi/yappi.egg-info/SOURCES.txt'
/tmp/timer_create8z57jb1h.c: In function ‘main’:
/tmp/timer_create8z57jb1h.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
    2 |     timer_create();
      |     ^~~~~~~~~~~~
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-310
copying yappi/yappi.py -> build/lib.linux-x86_64-cpython-310
running build_ext
building '_yappi' extension
creating build/temp.linux-x86_64-cpython-310
creating build/temp.linux-x86_64-cpython-310/yappi
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -fPIC -DLIB_RT_AVAILABLE=1 -I/usr/include/python3.10 -c yappi/_yappi.c -o build/temp.linux-x86_64-cpython-310/yappi/_yappi.o
In file included from yappi/_yappi.c:10:
yappi/config.h:4:10: fatal error: Python.h: No such file or directory
    4 | #include "Python.h"
      |          ^~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
Error occurs: 
Traceback (most recent call last):
  File "/usr/lib64/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 207, in install_candidate
    self.manager.install(can)
  File "/usr/lib/python3.10/site-packages/pdm/installers/manager.py", line 39, in install
    installer(prepared.build(), self.environment, prepared.direct_url())
  File "/usr/lib/python3.10/site-packages/pdm/models/candidates.py", line 331, in build
    self.wheel = builder.build(build_dir, metadata_directory=self._metadata_dir)
  File "/usr/lib/python3.10/site-packages/pdm/builders/wheel.py", line 28, in build
    filename = self._hook.build_wheel(out_dir, config_settings, metadata_directory)
  File "/usr/lib/python3.10/site-packages/pep517/wrappers.py", line 208, in build_wheel
    return self._call_hook('build_wheel', {
  File "/usr/lib/python3.10/site-packages/pep517/wrappers.py", line 322, in _call_hook
    self._subprocess_runner(
  File "/usr/lib/python3.10/site-packages/pdm/builders/base.py", line 246, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/usr/lib/python3.10/site-packages/pdm/builders/base.py", line 87, in log_subprocessor
    raise BuildError(
pdm.exceptions.BuildError: Call command ['/usr/bin/python3.10', '/usr/lib/python3.10/site-packages/pep517/in_process/_in_process.py', 'build_wheel', '/tmp/tmprefrneax'] return non-zero status(1).
Preparing isolated env for PEP 517 build...
Reusing shared build env: /tmp/pdm-build-env-eq22vk28-shared
Building wheel for https://files.pythonhosted.org/packages/9c/bb/47be36b473e56360d3012bf1b6e405785e42d1f2e91da715964c1a705937/yappi-1.3.5.tar.gz#sha256=f54c25f04aa7c613633b529bffd14e0699a4363f414dc9c065616fd52064a49b (from https://pypi.org/simple/yappi/)
/tmp/timer_createh4zqpscq.c: In function ‘main’:
/tmp/timer_createh4zqpscq.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
    2 |     timer_create();
      |     ^~~~~~~~~~~~
running egg_info
writing yappi/yappi.egg-info/PKG-INFO
writing dependency_links to yappi/yappi.egg-info/dependency_links.txt
writing entry points to yappi/yappi.egg-info/entry_points.txt
writing requirements to yappi/yappi.egg-info/requires.txt
writing top-level names to yappi/yappi.egg-info/top_level.txt
reading manifest file 'yappi/yappi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'yappi/yappi.egg-info/SOURCES.txt'
/tmp/timer_createnezrnkq7.c: In function ‘main’:
/tmp/timer_createnezrnkq7.c:2:5: warning: implicit declaration of function ‘timer_create’ [-Wimplicit-function-declaration]
    2 |     timer_create();
      |     ^~~~~~~~~~~~
running bdist_wheel
running build
running build_py
running build_ext
building '_yappi' extension
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -fPIC -DLIB_RT_AVAILABLE=1 -I/usr/include/python3.10 -c yappi/_yappi.c -o build/temp.linux-x86_64-cpython-310/yappi/_yappi.o
In file included from yappi/_yappi.c:10:
yappi/config.h:4:10: fatal error: Python.h: No such file or directory
    4 | #include "Python.h"
      |          ^~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
Error occurs: 
Traceback (most recent call last):
  File "/usr/lib64/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 207, in install_candidate
    self.manager.install(can)
  File "/usr/lib/python3.10/site-packages/pdm/installers/manager.py", line 39, in install
    installer(prepared.build(), self.environment, prepared.direct_url())
  File "/usr/lib/python3.10/site-packages/pdm/models/candidates.py", line 331, in build
    self.wheel = builder.build(build_dir, metadata_directory=self._metadata_dir)
  File "/usr/lib/python3.10/site-packages/pdm/builders/wheel.py", line 28, in build
    filename = self._hook.build_wheel(out_dir, config_settings, metadata_directory)
  File "/usr/lib/python3.10/site-packages/pep517/wrappers.py", line 208, in build_wheel
    return self._call_hook('build_wheel', {
  File "/usr/lib/python3.10/site-packages/pep517/wrappers.py", line 322, in _call_hook
    self._subprocess_runner(
  File "/usr/lib/python3.10/site-packages/pdm/builders/base.py", line 246, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/usr/lib/python3.10/site-packages/pdm/builders/base.py", line 87, in log_subprocessor
    raise BuildError(
pdm.exceptions.BuildError: Call command ['/usr/bin/python3.10', '/usr/lib/python3.10/site-packages/pep517/in_process/_in_process.py', 'build_wheel', '/tmp/tmpga2r2t_o'] return non-zero status(1).
Error occurs
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/pdm/termui.py", line 200, in logging
    yield logger
  File "/usr/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 374, in synchronize
    raise InstallationError("Some package operations are not complete yet")
pdm.exceptions.InstallationError: Some package operations are not complete yet
jordaneremieff commented 2 years ago

Can you confirm where in Mangum these dependencies are coming from? The only dependency specified in setup.py is typing_extensions other than that the requirements.txt file in the repo contains only dev dependencies not included or required in the installation package.

Fak3 commented 2 years ago

Oops, my bad. I got confused maGNum with maNGum. Maybe you add a note in the readme installation section? Its easy to mistype or misread these two