search

PelionIoT / manifest-tool

A tool for creating and parsing update manifests
Apache License 2.0
11 stars 14 forks source link

Compile error when installing with python 3.10.2 #33

Closed Vibrionales closed 1 year ago

Vibrionales commented 2 years ago

Problem

When trying to install manifest-tool with python 3.10.2 it exits the setup because gcc failed. (see error below) python ver.: 3.10.2 pip ver.: 21.2.3 gcc ver.: 11.2.1 Installing with python 3.8 does work.

Command line error

command line error ``` `Installing collected packages: manifest-tool Running setup.py install for manifest-tool ... error ERROR: Command errored out with exit status 1: command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yjw7bx_w/manifest-tool_f7bbb78c03ab4282b4d77d300a46b960/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yjw7bx_w/manifest-tool_f7bbb78c03ab4282b4d77d300a46b960/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3i6fei9c/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/wnijenhuis/.local/include/python3.10/manifest-tool cwd: /tmp/pip-install-yjw7bx_w/manifest-tool_f7bbb78c03ab4282b4d77d300a46b960/ Complete output (120 lines): running install running build running build_py creating build creating build/lib.linux-x86_64-3.10 creating build/lib.linux-x86_64-3.10/manifesttool copying manifesttool/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool creating build/lib.linux-x86_64-3.10/manifesttool/package_tool copying manifesttool/package_tool/package_tool.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool copying manifesttool/package_tool/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool creating build/lib.linux-x86_64-3.10/manifesttool/mtool copying manifesttool/mtool/payload_format.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool copying manifesttool/mtool/mtool.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool copying manifesttool/mtool/ecdsa_helper.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool copying manifesttool/mtool/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool creating build/lib.linux-x86_64-3.10/manifesttool/dev_tool copying manifesttool/dev_tool/dev_tool.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool copying manifesttool/dev_tool/defaults.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool copying manifesttool/dev_tool/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool creating build/lib.linux-x86_64-3.10/manifesttool/delta_tool copying manifesttool/delta_tool/delta_tool.py -> build/lib.linux-x86_64-3.10/manifesttool/delta_tool copying manifesttool/delta_tool/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/delta_tool creating build/lib.linux-x86_64-3.10/manifesttool/common copying manifesttool/common/common_helpers.py -> build/lib.linux-x86_64-3.10/manifesttool/common copying manifesttool/common/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/common creating build/lib.linux-x86_64-3.10/manifesttool/package_tool/package_format copying manifesttool/package_tool/package_format/tar_package.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/package_format copying manifesttool/package_tool/package_format/package_format.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/package_format copying manifesttool/package_tool/package_format/embedded_package.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/package_format copying manifesttool/package_tool/package_format/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/package_format creating build/lib.linux-x86_64-3.10/manifesttool/package_tool/asn1 copying manifesttool/package_tool/asn1/package_schema.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/asn1 copying manifesttool/package_tool/asn1/package_encoder.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/asn1 copying manifesttool/package_tool/asn1/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/asn1 creating build/lib.linux-x86_64-3.10/manifesttool/package_tool/actions copying manifesttool/package_tool/actions/parse.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/actions copying manifesttool/package_tool/actions/create.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/actions copying manifesttool/package_tool/actions/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/package_tool/actions creating build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1 copying manifesttool/mtool/asn1/manifest_codec.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1 copying manifesttool/mtool/asn1/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1 creating build/lib.linux-x86_64-3.10/manifesttool/mtool/actions copying manifesttool/mtool/actions/schema.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/actions copying manifesttool/mtool/actions/public_key.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/actions copying manifesttool/mtool/actions/parse.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/actions copying manifesttool/mtool/actions/create.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/actions copying manifesttool/mtool/actions/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/actions creating build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v3 copying manifesttool/mtool/asn1/v3/manifest_schema_v3.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v3 copying manifesttool/mtool/asn1/v3/manifest_encoder_v3.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v3 copying manifesttool/mtool/asn1/v3/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v3 creating build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v1 copying manifesttool/mtool/asn1/v1/manifest_schema_v1.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v1 copying manifesttool/mtool/asn1/v1/manifest_encoder_v1.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v1 copying manifesttool/mtool/asn1/v1/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/mtool/asn1/v1 creating build/lib.linux-x86_64-3.10/manifesttool/dev_tool/pelion copying manifesttool/dev_tool/pelion/pelion.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/pelion copying manifesttool/dev_tool/pelion/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/pelion creating build/lib.linux-x86_64-3.10/manifesttool/dev_tool/actions copying manifesttool/dev_tool/actions/update.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/actions copying manifesttool/dev_tool/actions/init.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/actions copying manifesttool/dev_tool/actions/create.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/actions copying manifesttool/dev_tool/actions/__init__.py -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/actions running egg_info writing manifest_tool.egg-info/PKG-INFO writing dependency_links to manifest_tool.egg-info/dependency_links.txt writing entry points to manifest_tool.egg-info/entry_points.txt writing requirements to manifest_tool.egg-info/requires.txt writing top-level names to manifest_tool.egg-info/top_level.txt reading manifest file 'manifest_tool.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'manifest_tool.egg-info/SOURCES.txt' copying manifesttool/package_tool/package-schema.json -> build/lib.linux-x86_64-3.10/manifesttool/package_tool copying manifesttool/mtool/manifest-input-schema.json -> build/lib.linux-x86_64-3.10/manifesttool/mtool copying manifesttool/dev_tool/actions/code_template.txt -> build/lib.linux-x86_64-3.10/manifesttool/dev_tool/actions running build_ext building 'manifesttool.armbsdiff' extension creating build/temp.linux-x86_64-3.10 creating build/temp.linux-x86_64-3.10/bsdiff gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DLZ4_MEMORY_USAGE=10 -Ibsdiff -I/usr/include/python3.10 -c bsdiff/bsdiff.c -o build/temp.linux-x86_64-3.10/bsdiff/bsdiff.o --std=c99 -O3 gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DLZ4_MEMORY_USAGE=10 -Ibsdiff -I/usr/include/python3.10 -c bsdiff/bsdiff_helper.c -o build/temp.linux-x86_64-3.10/bsdiff/bsdiff_helper.o --std=c99 -O3 bsdiff/bsdiff_helper.c: In function ‘do_diff’: bsdiff/bsdiff_helper.c:158:42: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 3 has type ‘int64_t’ {aka ‘long int’} [-Wformat=] 158 | "Wrote diff file %s, size %lld. Max undeCompressBuffer frame size was %lld, max deCompressBuffer frame size was %lld.\n", | ~~~^ | | | long long int | %ld 159 | delta_file, patch_file_size, max_frame_size, | ~~~~~~~~~~~~~~~ | | | int64_t {aka long int} bsdiff/bsdiff_helper.c:158:86: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 4 has type ‘int64_t’ {aka ‘long int’} [-Wformat=] 158 | "Wrote diff file %s, size %lld. Max undeCompressBuffer frame size was %lld, max deCompressBuffer frame size was %lld.\n", | ~~~^ | | | long long int | %ld 159 | delta_file, patch_file_size, max_frame_size, | ~~~~~~~~~~~~~~ | | | int64_t {aka long int} bsdiff/bsdiff_helper.c:158:128: warning: format ‘%lld’ expects argument of type ‘long long int’, but argument 5 has type ‘int64_t’ {aka ‘long int’} [-Wformat=] 158 | "Wrote diff file %s, size %lld. Max undeCompressBuffer frame size was %lld, max deCompressBuffer frame size was %lld.\n", | ~~~^ | | | long long int | %ld 159 | delta_file, patch_file_size, max_frame_size, 160 | max_deCompressBuffer_size | ~~~~~~~~~~~~~~~~~~~~~~~~~ | | | int64_t {aka long int} gcc -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DLZ4_MEMORY_USAGE=10 -Ibsdiff -I/usr/include/python3.10 -c bsdiff/bsdiff_python.c -o build/temp.linux-x86_64-3.10/bsdiff/bsdiff_python.o --std=c99 -O3 bsdiff/bsdiff_python.c:19:10: fatal error: Python.h: No such file or directory 19 | #include | ^~~~~~~~~~ compilation terminated. error: command '/usr/bin/gcc' failed with exit code 1 ---------------------------------------- ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yjw7bx_w/manifest-tool_f7bbb78c03ab4282b4d77d300a46b960/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yjw7bx_w/manifest-tool_f7bbb78c03ab4282b4d77d300a46b960/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-3i6fei9c/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers` ```
JanneKiiskila commented 1 year ago

Hi @Vibrionales,

We have not had a chance to update the tool for Python 3.10 yet and it seems that the Python ecosystem likes to change/break almost everthing while it moves up the versions. We would recommend using for example virtualenv with a Python 3.8 installed as a secondary Python installment and use that.

However, in your particular problem - sudo apt install python3.8-dev will likely resolve your issue, since it seems compiler cannot find the Python.h header file (which should come as part of of the -dev package). You will also probably need sudo apt install python3.8-distutils.

JanneKiiskila commented 1 year ago

It does actually work, I just installed it on Ubuntu 22.04 LTS release. I'm quite sure you're just missing python-dev package.

$ virtualenv ~/venv/3.10-manifest
created virtual environment CPython3.10.6.final.0-64 in 149ms
  creator CPython3Posix(dest=/home/jannek/venv/3.10-manifest, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/jannek/.local/share/virtualenv)
    added seed packages: pip==22.3.1, setuptools==65.6.3, wheel==0.38.4
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
$ source ~/venv/3.10-manifest/bin/activate
(3.10-manifest) jannek@jannek-P720:~$ pip install manifest-tool
Collecting manifest-tool
  Using cached manifest-tool-2.4.1.tar.gz (94 kB)
  Preparing metadata (setup.py) ... done
Collecting PyYAML<=5.4.1,>=4.2b1
  Using cached PyYAML-5.4.1-cp310-cp310-linux_x86_64.whl
Collecting asn1ate<=0.6.0,>=0.5
  Using cached asn1ate-0.6.0-py3-none-any.whl (29 kB)
Collecting cryptography<=3.4.8,>=2.5
  Using cached cryptography-3.4.8-cp36-abi3-manylinux_2_24_x86_64.whl (3.0 MB)
Collecting jsonschema<=3.2.0,>=2.6.0
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting pyasn1<=0.4.8,>=0.3.1
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting requests<=2.26.0,>=2.20.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting pyparsing>=2.0.0
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting cffi>=1.12
  Using cached cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
Collecting attrs>=17.4.0
  Using cached attrs-22.2.0-py3-none-any.whl (60 kB)
Requirement already satisfied: setuptools in ./venv/3.10-manifest/lib/python3.10/site-packages (from jsonschema<=3.2.0,>=2.6.0->manifest-tool) (65.6.3)
Collecting pyrsistent>=0.14.0
  Using cached pyrsistent-0.19.2-py3-none-any.whl (57 kB)
Collecting six>=1.11.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.13-py2.py3-none-any.whl (140 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Building wheels for collected packages: manifest-tool
  Building wheel for manifest-tool (setup.py) ... done
  Created wheel for manifest-tool: filename=manifest_tool-2.4.1-cp310-cp310-linux_x86_64.whl size=338398 sha256=2b2d611ffb68eae1efadeef72484a5a7149fa6e749311e6d65acc3e379b76055
  Stored in directory: /home/jannek/.cache/pip/wheels/66/3f/23/5814fb8d3b7bd12482db5e3a5747d9431244878d2da89b3380
Successfully built manifest-tool
Installing collected packages: pyasn1, urllib3, six, PyYAML, pyrsistent, pyparsing, pycparser, idna, charset-normalizer, certifi, attrs, requests, jsonschema, cffi, asn1ate, cryptography, manifest-tool
Successfully installed PyYAML-5.4.1 asn1ate-0.6.0 attrs-22.2.0 certifi-2022.12.7 cffi-1.15.1 charset-normalizer-2.0.12 cryptography-3.4.8 idna-3.4 jsonschema-3.2.0 manifest-tool-2.4.1 pyasn1-0.4.8 pycparser-2.21 pyparsing-3.0.9 pyrsistent-0.19.2 requests-2.26.0 six-1.16.0 urllib3-1.26.13