conda / conda-build

Commands and tools for building conda packages
https://docs.conda.io/projects/conda-build/
Other
380 stars 421 forks source link

env vars must be bytestrings on Windows #1412

Closed ocefpaf closed 7 years ago

ocefpaf commented 7 years ago

Some recipes are failing when conda-build tries to apply the patch:

Source cache directory is: C:\Miniconda\conda-bld\src_cache
Downloading source to cache: freexl-1.0.2.tar.gz
Downloading http://www.gaia-gis.it/gaia-sins/freexl-1.0.2.tar.gz
Success
Applying patch: u'C:\\projects\\conda-recipes\\recipes\\freexl\\makefile.vc.patch'
Traceback (most recent call last):
  File "C:\Miniconda\Scripts\conda-build-all-script.py", line 11, in <module>
    load_entry_point('conda-build-all==1.0.0', 'console_scripts', 'conda-build-all')()
  File "C:\Miniconda\lib\site-packages\conda_build_all\cli.py", line 90, in main
    b.main()
  File "C:\Miniconda\lib\site-packages\conda_build_all\builder.py", line 263, in main
    built_dist_location = self.build(meta, build_config)
  File "C:\Miniconda\lib\site-packages\conda_build_all\builder.py", line 200, in build
    conda_build.api.build(meta.meta, config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\api.py", line 83, in build
    need_source_download=need_source_download, config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 1004, in build_tree
    config=recipe_config)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 593, in build
    config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\render.py", line 87, in parse_or_try_download
    source.provide(metadata.path, metadata.get_section('source'), config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\source.py", line 509, in provide
    apply_patch(src_dir, join(recipe_dir, patch), config, git)
  File "C:\Miniconda\lib\site-packages\conda_build\source.py", line 468, in apply_patch
    check_call_env([patch] + patch_args, cwd=src_dir)
  File "C:\Miniconda\lib\site-packages\conda_build\utils.py", line 462, in check_call_env
    return _func_defaulting_env_to_os_environ(subprocess.check_call, *popenargs, **kwargs)
  File "C:\Miniconda\lib\site-packages\conda_build\utils.py", line 458, in _func_defaulting_env_to_os_environ
    return func(*popenargs, **kwargs)
  File "C:\Miniconda\lib\subprocess.py", line 536, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Miniconda\lib\subprocess.py",

Full log https://ci.appveyor.com/project/comtbot/conda-recipes/build/job/kxq68fcnidpdj2kr and recipe https://github.com/ioos/conda-recipes/tree/master/recipes/freexl

(Tried to fix this in https://github.com/conda/conda-build/pull/1377 and https://github.com/conda/conda-build/pull/1383 but gave up because I do not have a Windows machine to finish it :unamused:)

ocefpaf commented 7 years ago

@msarahan I still see this error with latest conda and conda-build. See https://github.com/ODM2/conda-recipes-ODM2/pull/44

msarahan commented 7 years ago

take 2: https://github.com/conda/conda-build/pull/1473 - hopefully that's a better fix.

ocefpaf commented 7 years ago

BTW I haven't looked at https://github.com/ODM2/conda-recipes-ODM2/pull/44#issuecomment-254862738 yet and I am not sure if that is a conda-build or a conda-build-all bug.

msarahan commented 7 years ago

On the surface, it looks like a conda-build bug. I could not reproduce it on my mac, though. =(

ocefpaf commented 7 years ago

I will investigate this further tomorrow. I have similar setups locally and in the IOOS channel, but only this one shows that error.

ocefpaf commented 7 years ago

@msarahan any chance of getting a new release? conda-forge's stack is broken on windows until the move to conda-build 2 happens due to https://github.com/conda-forge/hdf5-feedstock/issues/41 and defaults gdal is quite limited. So in order to put a working gdal for Windows user in another channel we really need the fix from #1473.

(cc @mwengren)

msarahan commented 7 years ago

Done - packages are built and should hopefully be available soon.

ocefpaf commented 7 years ago

Done - packages are built and should hopefully be available soon.

Thanks a lot @msarahan! Sorry to keep bothering you on this one. I promise to re-pay in :beers: next SciPy.

ocefpaf commented 7 years ago

conda-build-2.0.7 is the latest, right? I will try to set my WindowsVM today and dig into this a little bit. Maybe the problem is in that patch or conda-build-all.

Source cache directory is: C:\Miniconda\conda-bld\src_cache
Downloading source to cache: libiconv-1.14.tar.gz
Downloading http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.14.tar.gz
Success
Applying patch: u'C:\\projects\\conda-recipes\\recipes\\libiconv\\CMakeLists.txt.patch'
Traceback (most recent call last):
  File "C:\Miniconda\Scripts\conda-build-all-script.py", line 11, in <module>
    load_entry_point('conda-build-all==1.0.0', 'console_scripts', 'conda-build-all')()
  File "C:\Miniconda\lib\site-packages\conda_build_all\cli.py", line 90, in main
    b.main()
  File "C:\Miniconda\lib\site-packages\conda_build_all\builder.py", line 263, in main
    built_dist_location = self.build(meta, build_config)
  File "C:\Miniconda\lib\site-packages\conda_build_all\builder.py", line 200, in build
    conda_build.api.build(meta.meta, config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\api.py", line 77, in build
    need_source_download=need_source_download, config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 1099, in build_tree
    config=recipe_config)
  File "C:\Miniconda\lib\site-packages\conda_build\build.py", line 666, in build
    config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\render.py", line 86, in parse_or_try_download
    source.provide(metadata.path, metadata.get_section('source'), config=config)
  File "C:\Miniconda\lib\site-packages\conda_build\source.py", line 509, in provide
    apply_patch(src_dir, join(recipe_dir, patch), config, git)
  File "C:\Miniconda\lib\site-packages\conda_build\source.py", line 468, in apply_patch
    check_call_env([patch] + patch_args, cwd=src_dir)
  File "C:\Miniconda\lib\site-packages\conda_build\utils.py", line 552, in check_call_env
    return _func_defaulting_env_to_os_environ(subprocess.check_call, *popenargs, **kwargs)
  File "C:\Miniconda\lib\site-packages\conda_build\utils.py", line 548, in _func_defaulting_env_to_os_environ
    return func(_args, **kwargs)
  File "C:\Miniconda\lib\subprocess.py", line 536, in check_call
    retcode = call(*popenargs, **kwargs)
  File "C:\Miniconda\lib\subprocess.py", line 523, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Miniconda\lib\subprocess.py", line 711, in __init__
    errread, errwrite)
  File "C:\Miniconda\lib\subprocess.py", line 959, in _execute_child
    startupinfo)
TypeError: environment can only contain strings

Full log: https://ci.appveyor.com/project/comtbot/conda-recipes/build/job/o8amhafpvprqi06p

msarahan commented 7 years ago

2.0.7 is the latest. Sorry that didn't fix your issue. I thought it was settled and tested.

I'm teaching today and tomorrow, and have very little time to work on this. If you can help isolate it, that would be great.

ocefpaf commented 7 years ago

I'm teaching today and tomorrow, and have very little time to work on this. If you can help isolate it, that would be great.

Don't worry. I will see what I can do on my end. Will report back soon...

ocefpaf commented 7 years ago

Will report back soon...

@msarahan just to let you know conda-build 2 did apply the patch correctly in my local test. So the problem is probably be fixed on the conda-build side and the bug might be due to AppVeyor, conda-build-all, or something else. More soon...

ocefpaf commented 7 years ago

Just reporting back in case someone finds this thread. The issue went away by updating miniconda.

github-actions[bot] commented 2 years ago

Hi there, thank you for your contribution!

This issue has been automatically locked because it has not had recent activity after being closed.

Please open a new issue if needed.

Thanks!