Version preference breaks python-black in hardknott and dunfell

[priv-kweihmann]

Describe the bug

in hardknott and dunfell branch by default not the latest version of python3-setuptools-scm is used (provided by meta-sca) but the one shipped by poky (6.0.x). python3-black-native unfortunately requires a 6.3.x release.

| distutils.errors.DistutilsError: Command '['/opt/build/build/tmp/work/x86_64-linux/python3-black-native/21.10b0-r0/recipe-sysroot-native/usr/bin/python3-native/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp2mtyqib5', '--quiet', 'setuptools_scm[toml]>=6.3.1']' returned non-zero exit status 1.

It's actually okay to use PREFERRED_VERSION for these older releases to make an explicit choice towards the recipes provided by meta-sca. This version pinning should be done for the used distro setting

[priv-kweihmann]

Closed in honister/hardknott/dunfell

[fmartinsons]

Hello @priv-kweihmann

I fall into this exact same issue with dunfell. As suggested by the doc I set PREFERRED_VERSION_python3-setuptools-scm-native = "6.4.2" in my conf , but I encounter issue on python3-packaging recipe:

ERROR: python3-packaging-20.3-r0 do_compile: 'python3 setup.py build ' execution failed.
ERROR: python3-packaging-20.3-r0 do_compile: Execution of '/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/temp/run.do_compile.126332' failed with exit code 1
ERROR: Logfile of failure stored in: /home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/temp/log.do_compile.126332
Log data follows:
| DEBUG: Executing shell function do_compile
| Traceback (most recent call last):
|   File "setup.py", line 41, in <module>
|     setup(
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 143, in setup
|     _install_setup_requires(attrs)
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/__init__.py", line 131, in _install_setup_requires
|     dist = distutils.core.Distribution(dict(
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/dist.py", line 447, in __init__
|     _Distribution.__init__(self, {
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/distutils/dist.py", line 292, in __init__
|     self.finalize_options()
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/setuptools/dist.py", line 740, in finalize_options
|     ep.load()(self)
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2444, in load
|     self.require(*args, **kwargs)
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2467, in require
|     items = working_set.resolve(reqs, env, installer, extras=self.extras)
|   File "/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/recipe-sysroot-native/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 787, in resolve
|     raise DistributionNotFound(req, requirers)
| pkg_resources.DistributionNotFound: The 'six' distribution was not found and is required by packaging
| ERROR: 'python3 setup.py build ' execution failed.
| WARNING: exit code 1 from a shell command.
| ERROR: Execution of '/home/fmartinsons/TAPOS_build/build-testos/tmp/work/corei7-64-testos-linux/python3-packaging/20.3-r0/temp/run.do_compile.126332' failed with exit code 1
ERROR: Task (/home/fmartinsons/TAPOS_build/tapos/meta-openembedded/meta-python/recipes-devtools/python/python3-packaging_20.3.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 827 tasks of which 821 didn't need to be rerun and 1 failed.
NOTE: Build completion summary:
NOTE:   do_deploy_source_date_epoch: 100.0% sstate reuse(1 setscene, 0 scratch)
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 2 seconds

I also try to use a more up to date version of this recipe, but I fall into a cyclic dependency that I didn't quite understand. The second solution that is suggested in the doc (remove the culprit module) is not acceptable for me, I want to use flake8 and it seems black-native is a dependency to it.

Did you have, by any chance , another suggestion to make if you know another way ? Thanks in advance.

[priv-kweihmann]

@fmartinsons there might be a third option - in https://github.com/priv-kweihmann/meta-sca/blob/hardknott/recipes-sca/python3-flake8-sca-native/python3-flake8-sca-native.bb for instance you would just have to remove strict from the PACKAGECONFIG, as this is the only module that relies on black as a dependency. IMO a PACKAGECONFIG_pn-python3-flake8-sca-native_remove = "strict" should do it, you can also deactivate with the help of bbappend.

I think dropping flake8-strict is an acceptable trade-off to get out of this misery with poky python module versions and black

[fmartinsons]

Ok thank you very much, I'll try the bbappend way and I'll keep you posted here.

[fmartinsons]

I used the bbappend method and add PACKAGECONFIG_remove += "strict". As you told, it allowed me to build and execute flake8. Thanks for the insight @priv-kweihmann