search

holoviz-dev / pyctdev

Python packaging Common Tasks for Developers
BSD 3-Clause "New" or "Revised" License
10 stars 2 forks source link

doit package_build not working on Python > 3.7 #95

Open maximlt opened 2 years ago

maximlt commented 2 years ago

For a while I've observed than trying to build packages on Python > 3.7 just fails, on conda not pip. This is for instance the error I got when trying to build Lumen on Python 3.8.

Traceback (most recent call last):
  File "/usr/share/miniconda3/envs/test/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda_build/cli/main_build.py", line 488, in main
    execute(sys.argv[1:])
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda_build/cli/main_build.py", line 458, in execute
    action(recipe, config)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda_build/cli/main_build.py", line 409, in test_action
    return api.test(recipe, move_broken=False, config=config)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda_build/api.py", line 221, in test
    test_result = test(recipedir_or_package_or_metadata, config=config, move_broken=move_broken,
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda_build/build.py", line 2865, in test
    actions = environ.get_install_actions(metadata.config.test_prefix,
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda_build/environ.py", line 799, in get_install_actions
    actions = install_actions(prefix, index, specs, force=True)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda/common/io.py", line 88, in decorated
    return f(*args, **kwds)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda/plan.py", line 466, in install_actions
    specs = tuple(MatchSpec(spec) for spec in specs)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda/plan.py", line 466, in <genexpr>
    specs = tuple(MatchSpec(spec) for spec in specs)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda/models/match_spec.py", line 47, in __call__
    parsed = _parse_spec_str(spec_arg)
  File "/usr/share/miniconda3/envs/test/lib/python3.8/site-packages/conda/models/match_spec.py", line 706, in _parse_spec_str
    if build is None and test_str[-1] != '*':
IndexError: string index out of range
TaskFailed - taskid:package_build
Command failed: 'conda build -c pyviz/label/dev -c conda-forge conda.recipe/ -t --append-file conda.recipe//recipe_append--py38-unit-default-pkg.yaml && conda build purge' returned 1

After a long debugging session I could pin down the issue to the get_tox_python, which on Python 3.7 returns '3.7' but returns an empty string on later versions of Python.

https://github.com/pyviz-dev/pyctdev/blob/1ad27d99b1724fe5bee9bc35c07c96ede5776085/pyctdev/util.py#L39-L44

Not sure it's a bug in tox (which pyctdev vendors) or elsewhere, but at least now we know where the problem comes from! Fixing it though properly will require me to better understand how doit package_build works.

jbednar commented 2 years ago

Maybe upgrade tox?

maximlt commented 2 years ago

Given the comment above the failing line I'd rather try to find a direct solution to that, before attempting to upgrade tox which could break pyctdev in various ways.

     # TODO really doubt this is the right way 
     return toxconf.envconfigs[env].basepython.split("python")[1]

First I'll have to understand what was the intention with this code, and if it was effectively working. There's no emergency though, not until Python 3.7 in EOL.