search

mozilla-releng / build-mar

Utility for managing mar files
Mozilla Public License 2.0
8 stars 13 forks source link

Not compatible with construct-2.9 #47

Closed coreyfarrell closed 5 years ago

coreyfarrell commented 5 years ago

The python construct 2.9 module removed a bunch of API's, this utility cannot be used as-is on Fedora 28. sudo pip3 install mar && mar -h produces the following backtrace:

Traceback (most recent call last):
  File "/usr/local/bin/mar", line 7, in <module>
    from mardor.cli import main
  File "/usr/local/lib/python3.6/site-packages/mardor/cli.py", line 14, in <module>
    from mardor.reader import MarReader
  File "/usr/local/lib/python3.6/site-packages/mardor/reader.py", line 12, in <module>
    from mardor.format import mar
  File "/usr/local/lib/python3.6/site-packages/mardor/format.py", line 13, in <module>
    from construct import Computed
ImportError: cannot import name 'Computed'

I was able to work-around this by pinning to 'construct==2.8.22' in requirements.txt and setup.py, then using python3 setup.py build && python3 setup.py install --user.

Callek commented 5 years ago

I don't seem to have the same issue using py 3.6 and mar with construct 3.9 (see the following local log) I'm on ubuntu 18.10 locally but python itself is not distro.

callek@vela:~$ pyenv virtualenv 3.6.6 tmp
Using base prefix '/home/callek/.pyenv/versions/3.6.6'
New python executable in /home/callek/.pyenv/versions/3.6.6/envs/tmp/bin/python3.6
Also creating executable in /home/callek/.pyenv/versions/3.6.6/envs/tmp/bin/python
Installing setuptools, pip, wheel...done.
Looking in links: /tmp/tmp6kbuynq9
Requirement already satisfied: setuptools in /home/callek/.pyenv/versions/3.6.6/envs/tmp/lib/python3.6/site-packages (40.6.2)
Requirement already satisfied: pip in /home/callek/.pyenv/versions/3.6.6/envs/tmp/lib/python3.6/site-packages (18.1)
callek@vela:~$ pyenv activate tmp
pyenv-virtualenv: prompt changing will be removed from future release. configure `export PYENV_VIRTUALENV_DISABLE_PROMPT=1' to simulate the behavior.
(tmp) callek@vela:~$ pip install mar
Collecting mar
  Using cached https://files.pythonhosted.org/packages/ad/68/fad1d3f84f4490eeb9db3293afd415cb8277045af94e280431c218628091/mar-3.0.0-py2.py3-none-any.whl
Collecting construct (from mar)
Collecting cryptography (from mar)
  Using cached https://files.pythonhosted.org/packages/60/c7/99b33c53cf3f20a97a4c4bfd3ab66dcc93d99da0a97cc9597aa36ae6bb62/cryptography-2.4.2-cp34-abi3-manylinux1_x86_64.whl
Collecting backports.lzma (from mar)
Collecting click (from mar)
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Collecting asn1crypto (from mar)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting six>=1.4.1 (from cryptography->mar)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.7 (from cryptography->mar)
  Using cached https://files.pythonhosted.org/packages/6d/c0/47db8f624f3e4e2f3f27be03a93379d1ba16a1450a7b1aacfa0366e2c0dd/cffi-1.11.5-cp36-cp36m-manylinux1_x86_64.whl
Collecting idna>=2.1 (from cryptography->mar)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.7->cryptography->mar)
Installing collected packages: construct, asn1crypto, six, pycparser, cffi, idna, cryptography, backports.lzma, click, mar
Successfully installed asn1crypto-0.24.0 backports.lzma-0.0.13 cffi-1.11.5 click-7.0 construct-2.9.45 cryptography-2.4.2 idna-2.7 mar-3.0.0 pycparser-2.19 six-1.11.0
(tmp) callek@vela:~$ pip freeze
asn1crypto==0.24.0
backports.lzma==0.0.13
cffi==1.11.5
Click==7.0
construct==2.9.45
cryptography==2.4.2
idna==2.7
mar==3.0.0
pycparser==2.19
six==1.11.0
(tmp) callek@vela:~$ python
Python 3.6.6 (default, Nov  2 2018, 14:19:15) 
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from construct import Computed
>>> Computed.__doc__
'\n    Field computing a value from the context dictionary or some outer source like os.urandom or random module. Underlying byte stream is unaffected. The source can be non-deterministic.\n\n    Parsing and Building return the value returned by the context lambda (although a constant value can also be used). Size is defined as 0 because parsing and building does not consume or produce bytes into the stream.\n\n    :param func: context lambda or constant value\n\n    Can propagate any exception from the lambda, possibly non-ConstructError.\n\n    Example::\n        >>> d = Struct(\n        ...     "width" / Byte,\n        ...     "height" / Byte,\n        ...     "total" / Computed(this.width * this.height),\n        ... )\n        >>> d.build(dict(width=4,height=5))\n        b\'\\x04\\x05\'\n        >>> d.parse(b"12")\n        Container(width=49, height=50, total=2450)\n\n        >>> d = Computed(7)\n        >>> d.parse(b"")\n        7\n        >>> d = Computed(lambda ctx: 7)\n        >>> d.parse(b"")\n        7\n\n        >>> import os\n        >>> d = Computed(lambda ctx: os.urandom(10))\n        >>> d.parse(b"")\n        b\'\\x98\\xc2\\xec\\x10\\x07\\xf5\\x8e\\x98\\xc2\\xec\'\n    '
coreyfarrell commented 5 years ago

I'm sorry I was mistaken, I don't know why I thought Fedora came with construct-2.9. It apparently has construct-2.5, so when I tried to install mar initially the setup.py just accepted that. I'm not sure if the setup.py should specify a minimum version? If not sorry for the noise and sorry for making an incorrect statement.