search

spine-tools / Spine-Database-API

Database interface to Spine generic data model
https://www.tools-for-energy-system-modelling.org/
GNU Lesser General Public License v3.0
6 stars 5 forks source link

ERROR: Failed to build installable wheels for some pyproject.toml based projects (gdxcc) #441

Open itsmostafa opened 4 weeks ago

itsmostafa commented 4 weeks ago

Hello, when attempting to install via docker the official python image (I've tried versions 3.8, 3.9, 3.10, and 3.11), I'm getting the following error:

 => ERROR [rescalc 8/8] RUN pip install gdx2py>=2.1.1 --verbose                                                              1.0s 
------
 > [8/8] RUN pip install gdx2py>=2.1.1 --verbose:
0.565   Running command python setup.py egg_info
0.655   running egg_info
0.655   creating /tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info
0.656   writing /tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info/PKG-INFO
0.656   writing dependency_links to /tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info/dependency_links.txt
0.656   writing top-level names to /tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info/top_level.txt
0.656   writing manifest file '/tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info/SOURCES.txt'
0.660   reading manifest file '/tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info/SOURCES.txt'
0.661   reading manifest template 'MANIFEST.in'
0.661   warning: no files found matching '*.rst'
0.661   warning: no files found matching '*.txt'
0.661   warning: no previously-included files matching '*.py[co]' found anywhere in distribution
0.661   warning: no previously-included files matching '__pycache__' found anywhere in distribution
0.661   adding license file 'LICENSE'
0.661   writing manifest file '/tmp/pip-pip-egg-info-731zq1fm/gdxcc.egg-info/SOURCES.txt'
0.703   Running command python setup.py bdist_wheel
0.798   running bdist_wheel
0.805   running build
0.805   running build_py
0.806   creating build
0.806   creating build/lib.linux-aarch64-cpython-311
0.806   creating build/lib.linux-aarch64-cpython-311/gdxcc
0.806   copying gdxcc/gdxcc.py -> build/lib.linux-aarch64-cpython-311/gdxcc
0.807   copying gdxcc/__init__.py -> build/lib.linux-aarch64-cpython-311/gdxcc
0.807   running egg_info
0.807   writing gdxcc.egg-info/PKG-INFO
0.808   writing dependency_links to gdxcc.egg-info/dependency_links.txt
0.808   writing top-level names to gdxcc.egg-info/top_level.txt
0.809   reading manifest file 'gdxcc.egg-info/SOURCES.txt'
0.809   reading manifest template 'MANIFEST.in'
0.809   warning: no files found matching '*.rst'
0.809   warning: no files found matching '*.txt'
0.809   warning: no previously-included files matching '*.py[co]' found anywhere in distribution
0.810   warning: no previously-included files matching '__pycache__' found anywhere in distribution
0.810   adding license file 'LICENSE'
0.810   writing manifest file 'gdxcc.egg-info/SOURCES.txt'
0.810   /usr/local/lib/python3.11/site-packages/setuptools/command/build_py.py:215: _Warning: Package 'gdxcc.C' is absent from the `packages` configuration.
0.810   !!
0.810 
0.810           ********************************************************************************
0.810           ############################
0.811           # Package would be ignored #
0.811           ############################
0.811           Python recognizes 'gdxcc.C' as an importable package[^1],
0.811           but it is absent from setuptools' `packages` configuration.
0.811 
0.811           This leads to an ambiguous overall configuration. If you want to distribute this
0.811           package, please make sure that 'gdxcc.C' is explicitly added
0.811           to the `packages` configuration field.
0.811 
0.811           Alternatively, you can also rely on setuptools' discovery methods
0.811           (for example by using `find_namespace_packages(...)`/`find_namespace:`
0.811           instead of `find_packages(...)`/`find:`).
0.811 
0.811           You can read more about "package discovery" on setuptools documentation page:
0.811 
0.812           - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
0.812 
0.812           If you don't want 'gdxcc.C' to be distributed and are
0.812           already explicitly excluding 'gdxcc.C' via
0.812           `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
0.812           you can try to use `exclude_package_data`, or `include-package-data=False` in
0.812           combination with a more fine grained `package-data` configuration.
0.812 
0.812           You can read more about "package data files" on setuptools documentation page:
0.812 
0.812           - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
0.812 
0.812 
0.813           [^1]: For Python, any directory (with suitable naming) can be imported,
0.813                 even if it does not contain any `.py` files.
0.813                 On the other hand, currently there is no concept of package data
0.813                 directory, all directories are treated like packages.
0.813           ********************************************************************************
0.813 
0.813   !!
0.813     check.warn(importable)
0.814   copying gdxcc/gdxcc.i -> build/lib.linux-aarch64-cpython-311/gdxcc
0.814   creating build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gclgms.c -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gclgms.h -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gcmt.c -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gcmt.h -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gdxcc.c -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gdxcc.h -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   copying gdxcc/C/gdxcc_wrap.c -> build/lib.linux-aarch64-cpython-311/gdxcc/C
0.814   running build_ext
0.814   building '_gdxcc' extension
0.814   creating build/temp.linux-aarch64-cpython-311
0.814   creating build/temp.linux-aarch64-cpython-311/gdxcc
0.815   creating build/temp.linux-aarch64-cpython-311/gdxcc/C
0.815   gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPYPREFIXGDX -D_CRT_SECURE_NO_WARNINGS -Igdxcc/C -I/usr/local/include/python3.11 -c gdxcc/C/gclgms.c -o build/temp.linux-aarch64-cpython-311/gdxcc/C/gclgms.o -m64
0.815   gcc: error: unrecognized command-line option ‘-m64’
0.815   error: command '/usr/bin/gcc' failed with exit code 1
0.836   error: subprocess-exited-with-error
0.836   
0.836   × python setup.py bdist_wheel did not run successfully.
0.836   │ exit code: 1
0.836   ╰─> See above for output.
0.836   
0.836   note: This error originates from a subprocess, and is likely not a problem with pip.
0.837   full command: /usr/local/bin/python -u -c '
0.837   exec(compile('"'"''"'"''"'"'
0.837   # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
0.837   #
0.837   # - It imports setuptools before invoking setup.py, to enable projects that directly
0.837   #   import from `distutils.core` to work with newer packaging standards.
0.837   # - It provides a clear error message when setuptools is not installed.
0.837   # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
0.837   #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
0.837   #     manifest_maker: standard file '"'"'-c'"'"' not found".
0.837   # - It generates a shim setup.py, for handling setup.cfg-only projects.
0.837   import os, sys, tokenize
0.837   
0.837   try:
0.837       import setuptools
0.837   except ImportError as error:
0.837       print(
0.837           "ERROR: Can not execute `setup.py` since setuptools is not available in "
0.837           "the build environment.",
0.837           file=sys.stderr,
0.837       )
0.837       sys.exit(1)
0.837   
0.837   __file__ = %r
0.837   sys.argv[0] = __file__
0.837   
0.837   if os.path.exists(__file__):
0.837       filename = __file__
0.837       with tokenize.open(__file__) as f:
0.837           setup_py_code = f.read()
0.837   else:
0.837       filename = "<auto-generated setuptools caller>"
0.837       setup_py_code = "from setuptools import setup; setup()"
0.837   
0.837   exec(compile(setup_py_code, filename, "exec"))
0.837   '"'"''"'"''"'"' % ('"'"'/tmp/pip-install-szeb3ovz/gdxcc_5f4c97a03a224926b8b286673c289210/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' bdist_wheel -d /tmp/pip-wheel-nphccorg
0.837   cwd: /tmp/pip-install-szeb3ovz/gdxcc_5f4c97a03a224926b8b286673c289210/
0.837   ERROR: Failed building wheel for gdxcc
0.837   Running command python setup.py clean
0.903   running clean
0.905   removing 'build/temp.linux-aarch64-cpython-311' (and everything under it)
0.905   removing 'build/lib.linux-aarch64-cpython-311' (and everything under it)
0.906   'build/bdist.linux-aarch64' does not exist -- can't clean it
0.906   'build/scripts-3.11' does not exist -- can't clean it
0.906   removing 'build'
0.939 ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (gdxcc)

Here is my Dockerfile that errors out with the above:

FROM python:3.10

WORKDIR /app

COPY . /app

# Install server dependencies
RUN apt-get update &&  apt-get install -y \
    build-essential \
    gcc \
    libpq-dev \
    postgresql-server-dev-all \
    python3-dev

# Install python dependencies
RUN pip install --upgrade pip setuptools wheel
RUN pip install gdx2py>=2.1.1 --verbose

For more context, installing gdx2py directly on a local virtual environment on my machine works fine. I'm running on MacOS Sonoma 14.5 with arm64 architecture. It appears to be an issue relating to the os environment. Thanks

ptsavol commented 4 weeks ago

Not sure since I'm not a Mac user but maybe there is no compatible wheel available for your docker platform. Looks like gdxcc is built for the following platforms:

itsmostafa commented 4 weeks ago

Hi @PekkaSavolainen thank you for your quick response. None of those python versions have been successful for me. However after digging deeper, it seems that the gdxcc python package (a subdependency of Spine-Database-API) doesn't sufficiently support Docker running on a MacOS ARM64 architecture. Knowing this, I found a temporary solution by adding the --platform=linux/amd64 tag so that I can access the compatible wheel. Here's my updated Dockerfile:

FROM --platform=linux/amd64 python:3.10-slim

WORKDIR /app

COPY . /app

# Install server dependencies
RUN apt-get update &&  apt-get install -y \
    build-essential \
    gcc \
    libpq-dev

# Install python dependencies
RUN pip install --upgrade pip setuptools wheel
RUN pip install gdx2py>=2.1.1 --verbose

I hope this helps anyone who comes across a similar issue in the meantime.