search

pypa / hatch

Modern, extensible Python project management
https://hatch.pypa.io/latest/
MIT License
6.1k stars 309 forks source link

Several tests related to Python distributions fail on Fedora Linux on non-x86_64 architectures #1145

Closed musicinmybrain closed 11 months ago

musicinmybrain commented 11 months ago

I’m working on updating the hatch and python-hatchling packages in Fedora Rawhide to the latest versions (1.8.1 and 1.20.0, respectively). On architectures other than x86_64, several tests related to the new Python distribution support fail. Here’s an example from aarch64:

=================================== FAILURES ===================================
_________________________ TestGetInstalled.test_order __________________________

self = <tests.python.test_core.TestGetInstalled object at 0xffff861921e0>
temp_dir = Path('/tmp/tmpb_72kh9s')

    def test_order(self, temp_dir):
        manager = PythonManager(temp_dir)

        for name in DISTRIBUTIONS:
>           dist = get_distribution(name)

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_core.py:71: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.7', source = '', variant = ''

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)

        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)

        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'

        if not variant:
            variant = _get_default_variant(name, system, arch, abi)

        key = (system, arch, abi, variant)

        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.7' system='linux' arch='aarch64' abi='gnu' variant=''

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v4] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v4'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')

        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v4'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)

        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)

        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'

        if not variant:
            variant = _get_default_variant(name, system, arch, abi)

        key = (system, arch, abi, variant)

        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v4'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v3] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v3'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')

        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v3'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)

        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)

        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'

        if not variant:
            variant = _get_default_variant(name, system, arch, abi)

        key = (system, arch, abi, variant)

        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v3'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v2] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v2'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')

        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v2'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)

        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)

        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'

        if not variant:
            variant = _get_default_variant(name, system, arch, abi)

        key = (system, arch, abi, variant)

        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v2'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
___________________________ test_variants[linux-v1] ____________________________

platform = <hatch.utils.platform.Platform object at 0xffff87e470e0>
system = 'linux', variant = 'v1'

    @pytest.mark.parametrize(
        ('system', 'variant'),
        [
            ('windows', 'shared'),
            ('windows', 'static'),
            ('linux', 'v1'),
            ('linux', 'v2'),
            ('linux', 'v3'),
            ('linux', 'v4'),
        ],
    )
    def test_variants(platform, system, variant):
        if platform.name != system:
            pytest.skip(f'Skipping test for: {system}')

        with EnvVars({f'HATCH_PYTHON_VARIANT_{system.upper()}': variant}):
>           dist = get_distribution('3.11')

/builddir/build/BUILD/hatch-hatch-v1.8.1/tests/python/test_resolve.py:55: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = '3.11', source = '', variant = 'v1'

    def get_distribution(name: str, source: str = '', variant: str = '') -> Distribution:
        if source:
            return _get_distribution_class(source)(name, source)

        if name not in DISTRIBUTIONS:
            message = f'Unknown distribution: {name}'
            raise PythonDistributionUnknownError(message)

        arch = platform.machine().lower()
        if sys.platform == 'win32':
            system = 'windows'
            abi = 'msvc'
        elif sys.platform == 'darwin':
            system = 'macos'
            abi = ''
        else:
            system = 'linux'
            abi = 'gnu' if any(platform.libc_ver()) else 'musl'

        if not variant:
            variant = _get_default_variant(name, system, arch, abi)

        key = (system, arch, abi, variant)

        keys: dict[tuple, str] = DISTRIBUTIONS[name]
        if key not in keys:
            message = f'Could not find a default source for {name=} {system=} {arch=} {abi=} {variant=}'
>           raise PythonDistributionResolutionError(message)
E           hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v1'

/builddir/build/BUILDROOT/hatch-1.8.1-1.fc40.aarch64/usr/lib/python3.12/site-packages/hatch/python/resolve.py:153: PythonDistributionResolutionError
=============================== warnings summary ===============================
tests/backend/builders/test_sdist.py::TestBuildStandard::test_include_readme
tests/backend/builders/test_sdist.py::TestBuildStandard::test_project_file_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_include_license_files
tests/backend/builders/test_sdist.py::TestBuildStandard::test_no_strict_naming
tests/backend/builders/test_sdist.py::TestBuildStandard::test_readme_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_config_file_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_license_files_always_included
tests/backend/builders/test_sdist.py::TestBuildStandard::test_include_project_file
  /usr/lib64/python3.12/tarfile.py:2220: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/python/test_core.py::TestGetInstalled::test_order - hatch.errors...
FAILED tests/python/test_resolve.py::test_variants[linux-v4] - hatch.errors.P...
FAILED tests/python/test_resolve.py::test_variants[linux-v3] - hatch.errors.P...
FAILED tests/python/test_resolve.py::test_variants[linux-v2] - hatch.errors.P...
FAILED tests/python/test_resolve.py::test_variants[linux-v1] - hatch.errors.P...
=== 5 failed, 1617 passed, 98 skipped, 176 deselected, 8 warnings in 37.98s ====

Full build logs

builder-live.aarch64.txt builder-live.i386.txt builder-live.ppc64le.txt builder-live.s390x.txt

It seems like one problem may be that test_variants is testing for x86_64 feature levels, but using the host architecture. I haven’t investigated what’s going on with TestGetInstalled::test_order.

For now, I’m just skipping these tests on non-x86_64 platforms.

ofek commented 11 months ago

Thanks for bringing this to my attention! I don't quite know how to debug this

ofek commented 11 months ago

Would you mind pasting the output of those platform inspection standard library functions? I might just be looking for the wrong stuff.

musicinmybrain commented 11 months ago

Would you mind pasting the output of those platform inspection standard library functions? I might just be looking for the wrong stuff.

Is this what you’re looking for? (Fedora Rawhide, all primary architectures):

x86_64

+ /usr/bin/python3 -c 'import platform; print(repr(platform.machine()))'
'x86_64'
+ /usr/bin/python3 -c 'import platform; print(repr(platform.libc_ver()))'
('glibc', '2.38.9000')

i686

'i686'
('glibc', '2.38.9000')

ppc64le

'ppc64le'
('glibc', '2.38.9000')

s390x

's390x'
('glibc', '2.38.9000')

aarch64

'aarch64'
('glibc', '2.38.9000')
bnavigator commented 11 months ago

openSUSE:

# LegacyX86

hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='i686' abi='gnu' variant='v4'

# ARM
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv7l' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv6l' abi='gnu' variant='v4'

# PowerPC
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='ppc64le' abi='gnu' variant='v4'

# RISCV
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='riscv64' abi='gnu' variant='v4'

# zSystems
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='s390x' abi='gnu' variant='v4'
ofek commented 11 months ago

Please let me know if this is now fixed

edit: specifically @bnavigator

hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='i686' abi='gnu' variant='v4'

There are no 32-bit distributions that are available so this will not be fixed. Even when there are official compiled distributions I'm not sure PSF would invest in maintenance of that architecture either.

hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='aarch64' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv7l' abi='gnu' variant='v4'
hatch.errors.PythonDistributionResolutionError: Could not find a default source for name='3.11' system='linux' arch='armv6l' abi='gnu' variant='v4'

This I didn't fix actually I just realized. Can you please explain the situation in which platform.machine() would not return aarch64?

bnavigator commented 11 months ago

There are still thousands of Raspberry Pis and similar barebones with 32-bit ARM out there. The official recommended Raspberry Pi OS ist still 32-bit: https://www.raspberrypi.com/software/operating-systems/#raspberry-pi-os-32-bit

Also check

There was a survey for openSUSE about ARMv7 use cases in 2020(!): https://en.opensuse.org/Portal:15.3/Surveys/ARM_Usecases

bnavigator commented 11 months ago

On Intel 32bit (ix86), aarch64, ppc64le, and s390x, the unit test error now shifted to:

[   64s] _______________________ TestErrors.test_resolution_error _______________________
[   64s] 
[   64s] self = <tests.python.test_resolve.TestErrors object at 0xf57404a8>
[   64s] platform = <hatch.utils.platform.Platform object at 0xf676ae80>
[   64s] 
[   64s]     @pytest.mark.skipif(sys.platform == 'darwin', reason='No variants for macOS')
[   64s]     def test_resolution_error(self, platform):
[   64s]         with EnvVars({f'HATCH_PYTHON_VARIANT_{platform.name.upper()}': 'foo'}), pytest.raises(
[   64s]             PythonDistributionResolutionError,
[   64s]             match=f"Could not find a default source for name='3.11' system='{platform.name}' arch=",
[   64s]         ):
[   64s] >           get_distribution('3.11')
[   64s] E           Failed: DID NOT RAISE <class 'hatch.errors.PythonDistributionResolutionError'>
[   64s] 
[   64s] /home/abuild/rpmbuild/BUILD/hatch-hatch-v1.9.1/tests/python/test_resolve.py:21: Failed
[   64s] =========================== short test summary info ============================
[   64s] FAILED tests/python/test_resolve.py::TestErrors::test_resolution_error - Fail...
[   64s] ========== 1 failed, 1799 passed, 107 skipped, 3 deselected in 46.85s ==========

armv7 still fails in the mentioned and additional tests, all related to the distributions choice:

[   64s] ________________________ ERROR at setup of test_binary _________________________
[   64s] 
[   64s] compatible_python_distributions = ()
[   64s] 
[   64s]     @pytest.fixture
[   64s]     def dist_name(compatible_python_distributions):
[   64s] >       return secrets.choice(compatible_python_distributions)
[   64s] 
[   64s] /home/abuild/rpmbuild/BUILD/hatch-hatch-v1.9.1/tests/cli/python/conftest.py:30: 
[   64s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
[   64s] 
[   64s] self = <random.SystemRandom object at 0x1d720b0>, seq = ()
[   64s] 
[   64s]     def choice(self, seq):
[   64s]         """Choose a random element from a non-empty sequence."""
[   64s]         # raises IndexError if seq is empty
[   64s] >       return seq[self._randbelow(len(seq))]
[   64s] E       IndexError: tuple index out of range
[   64s] 
[   64s] /usr/lib/python3.9/random.py:346: IndexError
...
[   65s] FAILED tests/cli/python/test_remove.py::test_basic - hatch.errors.PythonDistr...
[   65s] FAILED tests/cli/python/test_remove.py::test_all - hatch.errors.PythonDistrib...
[   65s] FAILED tests/cli/python/test_show.py::test_nothing_installed - AssertionError...
[   65s] FAILED tests/cli/python/test_show.py::test_all_installed - AssertionError: as...
[   65s] FAILED tests/cli/python/test_update.py::test_all - hatch.errors.PythonDistrib...
[   65s] FAILED tests/python/test_core.py::TestGetInstalled::test_not_a_directory - ha...
[   65s] FAILED tests/python/test_core.py::TestGetInstalled::test_no_metadata_file - h...
[   65s] FAILED tests/python/test_core.py::TestGetInstalled::test_no_python_path - hat...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v1] - hatch.errors.P...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v2] - hatch.errors.P...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v3] - hatch.errors.P...
[   65s] FAILED tests/python/test_resolve.py::test_variants[linux-v4] - hatch.errors.P...
[   65s] ERROR tests/cli/python/test_find.py::test_binary - IndexError: tuple index ou...
[   65s] ERROR tests/cli/python/test_find.py::test_parent - IndexError: tuple index ou...
[   65s] ERROR tests/cli/python/test_install.py::test_incompatible_single - IndexError...
[   65s] ERROR tests/cli/python/test_install.py::test_already_installed_latest - Index...
[   65s] ERROR tests/cli/python/test_install.py::test_already_installed_update_disabled
[   65s] ERROR tests/cli/python/test_install.py::test_already_in_path[in_current_path]
[   65s] ERROR tests/cli/python/test_install.py::test_already_in_path[in_new_path] - I...
[   65s] ERROR tests/cli/python/test_install.py::test_private - IndexError: tuple inde...
[   65s] ERROR tests/cli/python/test_install.py::test_specific_location - IndexError: ...
[   65s] ERROR tests/cli/python/test_remove.py::test_specific_location - IndexError: t...
[   65s] ERROR tests/cli/python/test_show.py::test_some_installed - IndexError: tuple ...
[   65s] ERROR tests/cli/python/test_show.py::test_specific_location - IndexError: tup...
[   65s] ERROR tests/cli/python/test_show.py::test_outdated - IndexError: tuple index ...
[   65s] ERROR tests/cli/python/test_update.py::test_basic - IndexError: tuple index o...
[   65s] ERROR tests/cli/python/test_update.py::test_specific_location - IndexError: t...

~If you have an openSUSE buildservice account~, you can see the build and test suite outputs here: https://build.opensuse.org/package/show/home:bnavigator:branches:hatchdebug/python-hatch

ofek commented 11 months ago

I fixed the one test that was failing https://github.com/pypa/hatch/pull/1177

Unfortunately I have no way to fix the ARM 32-bit tests because there is no available distribution https://github.com/indygreg/python-build-standalone/releases

musicinmybrain commented 11 months ago

On Intel 32bit (ix86), aarch64, ppc64le, and s390x, the unit test error now shifted to:


[   64s] _______________________ TestErrors.test_resolution_error _______________________

Fedora no longer supports 32-bit ARM, so I can’t comment on that, but on 32-bit x86 (i686) I do see the same thing @bnavigator reported on 1.9.1, and I can confirm it’s fixed by https://github.com/pypa/hatch/pull/1177. With that patch, the tests pass on all of our primary architectures.