search

pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.46k stars 3k forks source link

UnicodeDecodeError: invalid start byte in METADATA file at path #9866

Closed L-Belluomini closed 3 years ago

L-Belluomini commented 3 years ago

Description

fresh install of pi OS lite, pip dose not come installed bu python does. installing FreeTAKserver everything worked, after reboot both FreeTAKserver and pip broke. tryng to update pip showed me the ( WARNING: Value for scheme.platlib does not match. Please report this to https://github.com/pypa/pip/issues/9617 distutils: /usr/local/lib/python3.7/dist-packages ) so I found the issue, but found it so closed, so I opened this one, Hope is the right thing.

Expected behavior

No response

pip version

python -m pip --version pip 21.1 from /home/pi/.local/lib/python3.7/site-packages/pip

Python version

python 3.7

OS

Pi OS lite

How to Reproduce

-1. install pip using apt ( now I now this might be the culprit) https://pip.pypa.io/en/stable/installing/ ( following this guide I chose to do so)

  1. Get package from using pip FreeTAKserver[ui]
  2. Then run rebbot
  3. An error occurs.

Output

sudo python3 -m FreeTAKServer.controllers.services.FTS
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.7/dist-packages/FreeTAKServer/controllers/services/FTS.py", line 5, in <module>
    from FreeTAKServer.controllers.services.TCPDataPackageService import TCPDataPackageService as TCPFlaskFunctions
  File "/usr/local/lib/python3.7/dist-packages/FreeTAKServer/controllers/services/TCPDataPackageService.py", line 1, in <module>
    from .DataPackageServer import FlaskFunctions, Path, dp_directory, os, app, eventlet, const
  File "/usr/local/lib/python3.7/dist-packages/FreeTAKServer/controllers/services/DataPackageServer.py", line 13, in <module>
    from FreeTAKServer.controllers.DatabaseControllers.DatabaseController import DatabaseController
  File "/usr/local/lib/python3.7/dist-packages/FreeTAKServer/controllers/DatabaseControllers/DatabaseController.py", line 1, in <module>
    import FreeTAKServer.controllers.DatabaseControllers.table_controllers
  File "/usr/local/lib/python3.7/dist-packages/FreeTAKServer/controllers/DatabaseControllers/table_controllers.py", line 1, in <module>
    from sqlalchemy import text
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/__init__.py", line 10, in <module>
    from .schema import BLANK_SCHEMA  # noqa
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/schema.py", line 12, in <module>
    from .sql.base import SchemaVisitor  # noqa
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/__init__.py", line 8, in <module>
    from .expression import Alias  # noqa
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/expression.py", line 90, in <module>
    from .dml import Delete  # noqa
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/dml.py", line 12, in <module>
    from . import util as sql_util
  File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/sql/util.py", line 34, in <module>
    from .schema import Column
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 724, in exec_module
  File "<frozen importlib._bootstrap_external>", line 857, in get_code
  File "<frozen importlib._bootstrap_external>", line 525, in _compile_bytecode
EOFError: marshal data too short

python3 -m pip check FreeTAKserver
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
ERROR: Exception:
Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3012, in _parsed_pkg_info
    return self._pkg_info
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
    status = self.run(options, args)
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_internal/commands/check.py", line 25, in run
    package_set, parsing_probs = create_package_set_from_installed()
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 45, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2736, in requires
    dm = self._dep_map
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3032, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3014, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1424, in get_metadata
    return value.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte in METADATA file at path: /usr/local/lib/python3.7/dist-packages/greenlet-1.0.0.dist-info/METADATA

python -m pip install -U pip
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
WARNING: Value for scheme.platlib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/lib/python3.7/dist-packages
sysconfig: /usr/lib/python3.7/site-packages
WARNING: Value for scheme.purelib does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/lib/python3.7/dist-packages
sysconfig: /usr/lib/python3.7/site-packages
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/include/python3.7/UNKNOWN
sysconfig: /usr/include/python3.7m
WARNING: Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local/bin
sysconfig: /usr/bin
WARNING: Value for scheme.data does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /usr/local
sysconfig: /usr
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: pip in ./.local/lib/python3.7/site-packages (21.1)
ERROR: Error while checking for conflicts. Please file an issue on pip's issue tracker: https://github.com/pypa/pip/issues/new
Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3012, in _parsed_pkg_info
    return self._pkg_info
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 515, in _determine_conflicts
    return check_install_conflicts(to_install)
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 103, in check_install_conflicts
    package_set, _ = create_package_set_from_installed()
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_internal/operations/check.py", line 45, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2736, in requires
    dm = self._dep_map
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3023, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3032, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3014, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "/home/pi/.local/lib/python3.7/site-packages/pip/_vendor/pkg_resources/__init__.py", line 1424, in get_metadata
    return value.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte in METADATA file at path: /usr/local/lib/python3.7/dist-packages/greenlet-1.0.0.dist-info/METADATA
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/9617>
distutils: /home/pi/.local/include/python3.7m/UNKNOWN
sysconfig: /home/pi/.local/include/python3.7
WARNING: Additional context:
user = True
home = None
root = None
prefix = None

Code of Conduct

uranusjr commented 3 years ago

Your issue is not related to #9617. Greenlet has a broken METADATA; how did you install the package?

L-Belluomini commented 3 years ago

@uranusjr thanks for the correction. I installed the package using pip ( sudo python3 -m pip install FreeTAKServer[ui] ).

uranusjr commented 3 years ago

Try

sudo python3 -m pip install --ignore-installed greenlet==1.0.0

If it’s possible, it would be awesome if you could upload /usr/local/lib/python3.7/dist-packages/greenlet-1.0.0.dist-info/METADATA to somewhere if the above does not fix it for you.

L-Belluomini commented 3 years ago

I wasn't sure if you meant ( sudo python3 -m pip install --ignore-installed greenlet==1.0.0 ) or ( sudo python3 -m pip install FreeTakServer --ignore-installed greenlet==1.0.0 ) so I ran both in this order, both failed launching exception. METADATA https://drive.google.com/drive/folders/1SO88VvH5jXDqR_xMh865yHtTMTPAh7m3?usp=sharing

 sudo python3 -m pip install --ignore-installed greenlet
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting greenlet
  Using cached https://www.piwheels.org/simple/greenlet/greenlet-1.0.0-cp37-cp37m-linux_armv7l.whl
Exception:
Traceback (most recent call last):
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2962, in _dep_map
    return self.__dep_map
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2756, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2953, in _parsed_pkg_info
    return self._pkg_info
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2756, in __getattr__
    raise AttributeError(attr)
AttributeError: _pkg_info

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 369, in run
    self._warn_about_conflicts(to_install)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 495, in _warn_about_conflicts
    package_set, _dep_info = check_install_conflicts(to_install)
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/check.py", line 98, in check_install_conflicts
    package_set = create_package_set_from_installed()
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/check.py", line 41, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2700, in requires
    dm = self._dep_map
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2964, in _dep_map
    self.__dep_map = self._compute_dependencies()
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2973, in _compute_dependencies
    for req in self._parsed_pkg_info.get_all('Requires-Dist') or []:
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2955, in _parsed_pkg_info
    metadata = self.get_metadata(self.PKG_INFO)
  File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 1413, in get_metadata
    return value.decode('utf-8') if six.PY3 else value
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

sudo python3 -m pip install FreeTAKserver[ui] --ignore-installed greenlet
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting FreeTAKserver[ui]
  Using cached https://files.pythonhosted.org/packages/ea/63/96ae7af96b0abc813509bd936e524758d224e6f8a053f651ca946b45ef59/FreeTAKServer-1.7.5-py3-none-any.whl
Collecting greenlet
  Using cached https://www.piwheels.org/simple/greenlet/greenlet-1.0.0-cp37-cp37m-linux_armv7l.whl
Collecting jinja2>=2.11.3 (from FreeTAKserver[ui])
  Using cached https://files.pythonhosted.org/packages/7e/c2/1eece8c95ddbc9b1aeb64f5783a9e07a286de42191b7204d67b7496ddf35/Jinja2-2.11.3-py2.py3-none-any.whl
Collecting lxml>=4.6.3 (from FreeTAKserver[ui])
Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/_internal/commands/install.py", line 338, in run
    resolver.resolve(requirement_set)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolve.py", line 102, in resolve
    self._resolve_one(requirement_set, req)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/lib/python3/dist-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/usr/lib/python3/dist-packages/pip/_internal/operations/prepare.py", line 283, in prepare_linked_requirement
    progress_bar=self.progress_bar
  File "/usr/lib/python3/dist-packages/pip/_internal/download.py", line 836, in unpack_url
    progress_bar=progress_bar
  File "/usr/lib/python3/dist-packages/pip/_internal/download.py", line 673, in unpack_http_url
    progress_bar)
  File "/usr/lib/python3/dist-packages/pip/_internal/download.py", line 868, in _download_http_url
    stream=True,
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/pip/_internal/download.py", line 396, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 37, in send
    cached_response = self.controller.cached_request(request)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/controller.py", line 111, in cached_request
    resp = self.serializer.loads(request, cache_data)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/serialize.py", line 114, in loads
    return getattr(self, "_loads_v{0}".format(ver))(request, data)
  File "/usr/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/serialize.py", line 176, in _loads_v2
    cached = json.loads(zlib.decompress(data).decode("utf8"))
zlib.error: Error -3 while decompressing data: invalid block type
uranusjr commented 3 years ago

Looks like the file is entirely broken, not sure how it ever got there. Try deleting the entire .dist-info directory:

rm -rf /usr/local/lib/python3.7/dist-packages/greenlet-1.0.0.dist-info

and try again.

uranusjr commented 3 years ago

Closing since this is not technically an issue in pip, but environmental and specific to the user.