conda-tools / conda-build-all

BSD 3-Clause "New" or "Revised" License
30 stars 24 forks source link

Building packages without source #51

Closed jakirkham closed 8 years ago

jakirkham commented 8 years ago

From @jakirkham on July 25, 2016 4:4

Some packages don't have a source section. Recently I have been having some issues with a package that has no source. However, have had some difficulties producing it offline with conda-build alone. So it may be a conda-build-all issue in the end. Here is a log demonstrating this failure. A traceback is shown below. It seems that maybe there are some cases where the work directory is not removed when it is expected to have been.

no source - creating empty work folder
Traceback (most recent call last):
  File "/opt/conda/bin/conda-build-all", line 9, in <module>
    load_entry_point('conda-build-all==0.13.1', 'console_scripts', 'conda-build-all')()
  File "/opt/conda/lib/python3.5/site-packages/conda_build_all/cli.py", line 85, in main
    b.main()
  File "/opt/conda/lib/python3.5/site-packages/conda_build_all/builder.py", line 230, in main
    built_dist_location = self.build(meta)
  File "/opt/conda/lib/python3.5/site-packages/conda_build_all/builder.py", line 187, in build
    return bldpkg_path(build.build(meta.meta))
  File "/opt/conda/lib/python3.5/site-packages/conda_build_all/build.py", line 32, in build
    build_module.build(meta, post=None, need_source_download=True, **kwd)
  File "/opt/conda/lib/python3.5/site-packages/conda_build/build.py", line 548, in build
    os.makedirs(source.WORK_DIR)
  File "/opt/conda/lib/python3.5/os.py", line 241, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/opt/conda/conda-bld/work'

cc @msarahan @pelson

Copied from original issue: conda/conda-build#1145

jakirkham commented 8 years ago

Sure enough I can reproduce this with conda-build-all alone. Build with conda-build proceeds without issue. Created the following recipe.

package:
  name: test
  version: 1.0.0

build:
  number: 0
  script: echo "test"

Here is the result of the build.

$ conda-build-all ./recipes
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata ...............
Resolving distributions from 1 recipes... 
Computed that there are 1 distributions from the 1 recipes:
Resolved dependencies, will be built in the following order: 
    test-1.0.0-0 (will be built: True)
Building  test-1.0.0-0
Removing old build environment
BUILD START: test-1.0.0-0
    (actual version deferred until further download or env creation)
BUILD START: test-1.0.0-0
Package: test-1.0.0-0
no source - creating empty work folder
Traceback (most recent call last):
  File "/zopt/conda3/bin/conda-build-all", line 9, in <module>
    load_entry_point('conda-build-all==0.13.1', 'console_scripts', 'conda-build-all')()
  File "/zopt/conda3/lib/python3.5/site-packages/conda_build_all/cli.py", line 85, in main
    b.main()
  File "/zopt/conda3/lib/python3.5/site-packages/conda_build_all/builder.py", line 230, in main
    built_dist_location = self.build(meta)
  File "/zopt/conda3/lib/python3.5/site-packages/conda_build_all/builder.py", line 187, in build
    return bldpkg_path(build.build(meta.meta))
  File "/zopt/conda3/lib/python3.5/site-packages/conda_build_all/build.py", line 32, in build
    build_module.build(meta, post=None, need_source_download=True, **kwd)
  File "/zopt/conda3/lib/python3.5/site-packages/conda_build/build.py", line 548, in build
    os.makedirs(source.WORK_DIR)
  File "/zopt/conda3/lib/python3.5/os.py", line 241, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/zopt/conda3/conda-bld/work'

Also, here is some info about my configuration.

$ conda info
Current conda install:

             platform : osx-64
        conda version : 4.1.9
    conda-env version : 2.5.2
  conda-build version : 1.21.7
       python version : 3.5.2.final.0
     requests version : 2.10.0
     root environment : /zopt/conda3  (writable)
  default environment : /zopt/conda3
     envs directories : /zopt/conda3/envs
        package cache : /zopt/conda3/pkgs
         channel URLs : https://conda.anaconda.org/t/<TOKEN>/conda-forge/osx-64/
                        https://conda.anaconda.org/t/<TOKEN>/conda-forge/noarch/
                        https://conda.anaconda.org/t/<TOKEN>/nanshe/osx-64/
                        https://conda.anaconda.org/t/<TOKEN>/nanshe/noarch/
                        https://repo.continuum.io/pkgs/free/osx-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/osx-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
                        https://conda.anaconda.org/t/<TOKEN>/jakirkham/osx-64/
                        https://conda.anaconda.org/t/<TOKEN>/jakirkham/noarch/
                        https://conda.anaconda.org/t/<TOKEN>/javascript/osx-64/
                        https://conda.anaconda.org/t/<TOKEN>/javascript/noarch/
          config file : /Users/kirkhamj/.condarc
         offline mode : False
    is foreign system : False

$ conda list
# packages in environment at /zopt/conda3:
#
Using Anaconda Cloud api site https://api.anaconda.org
anaconda-client           1.4.0                    py35_0    defaults
appnope                   0.1.0                    py35_0    defaults
conda                     4.1.9                    py35_0    defaults
conda-build               1.21.7                   py35_0    defaults
backports.shutil_get_terminal_size 1.0.0                    py35_0    conda-forge
ca-certificates           2016.2.28                     0    conda-forge
certifi                   2016.2.28                py35_1    conda-forge
clyent                    1.2.1                    py35_0    conda-forge
conda-build-all           0.13.1                   py35_0    conda-forge
conda-env                 2.5.2                    py35_0    conda-forge
decorator                 4.0.10                   py35_0    conda-forge
freetype                  2.6.3                         1    conda-forge
gitdb                     0.6.4                    py35_0    conda-forge
gitpython                 1.0.1                    py35_0    conda-forge
ipython                   5.0.0                    py35_0    conda-forge
jinja2                    2.8                      py35_1    conda-forge
jpeg                      9b                            0    conda-forge
libpng                    1.6.23                        0    conda-forge
libtiff                   4.0.6                         6    conda-forge
markupsafe                0.23                     py35_0    conda-forge
mock                      2.0.0                    py35_0    conda-forge
ncurses                   5.9                           7    conda-forge
openssl                   1.0.2h                        1    conda-forge
pexpect                   4.1.0                    py35_0    conda-forge
pickleshare               0.7.3                    py35_0    conda-forge
pillow                    3.2.0                    py35_2    conda-forge
pip                       8.1.2                    py35_0    conda-forge
prompt_toolkit            1.0.3                    py35_0    conda-forge
ptyprocess                0.5.1                    py35_0    conda-forge
pycosat                   0.6.1                    py35_0    conda-forge
pycrypto                  2.6.1                    py35_0    conda-forge
pygithub                  1.26.0                   py35_0    conda-forge
pygments                  1.6                      py35_0    conda-forge
python                    3.5.2                         1    conda-forge
python-dateutil           2.5.2                    py35_0    conda-forge
python-simplegeneric      0.8.1                    py35_0    conda-forge
pytz                      2016.3                   py35_0    conda-forge
pyyaml                    3.11                     py35_0    conda-forge
readline                  6.2                           0    conda-forge
requests                  2.10.0                   py35_0    conda-forge
ruamel.yaml               0.11.11                  py35_1    conda-forge
setuptools                23.0.0                   py35_1    conda-forge
simplegeneric             0.8.1                    py35_0    conda-forge
six                       1.10.0                   py35_0    conda-forge
smmap                     0.9.0                    py35_0    conda-forge
sqlite                    3.13.0                        1    conda-forge
tk                        8.5.19                        0    conda-forge
traitlets                 4.2.2                    py35_0    conda-forge
wcwidth                   0.1.7                    py35_0    conda-forge
wheel                     0.29.0                   py35_0    conda-forge
xz                        5.2.2                         0    conda-forge
yaml                      0.1.6                         0    conda-forge
zlib                      1.2.8                         3    conda-forge
ipython_genutils          0.1.0                    py35_0    defaults
jbig                      2.1                           0    defaults
path.py                   8.2.1                    py35_0    defaults
pbr                       1.10.0                   py35_0    defaults
python.app                1.2                      py35_4    defaults
ruamel_yaml               0.11.7                   py35_0    defaults
jakirkham commented 8 years ago

Some other use cases include PR ( https://github.com/conda-forge/staged-recipes/pull/1144 ) where they need a package that does not contain source or metapackages like blas.

pelson commented 8 years ago

Looks like the existing integration tests should already capture this failing. It is only occurring on py3.

jakirkham commented 8 years ago

So should this be solved by the recent release of conda-build plus other changes in the new conda-build-all release.

pelson commented 8 years ago

I believe so. All tests are passing on py27 and py35.

jakirkham commented 8 years ago

Ok, closing this out. We can always reopen if we re-encounter these issues.

jakirkham commented 8 years ago

Thanks for helping track this one down, @pelson. 😄