KeyError: 'python-1.0.1-0.tar.bz2' from rerender

[asmeurer]

No circle token.  Create a token at https://circleci.com/account/api and
put it in ~/.conda-smithy/circle.token
No appveyor token. Create a token at https://ci.appveyor.com/api-token and
Put one in ~/.conda-smithy/appveyor.token
No anaconda token. Create a token via
  anaconda auth --create --name conda-smithy --scopes "repos conda api"
and put it in ~/.conda-smithy/anaconda.token
Fetching package metadata .................
Traceback (most recent call last):
  File "/Users/aaronmeurer/anaconda3/bin/conda-smithy", line 6, in <module>
    sys.exit(conda_smithy.cli.main())
  File "/Users/aaronmeurer/anaconda3/lib/python3.5/site-packages/conda_smithy/cli.py", line 289, in main
    args.subcommand_func(args)
  File "/Users/aaronmeurer/anaconda3/lib/python3.5/site-packages/conda_smithy/cli.py", line 200, in __call__
    configure_feedstock.main(args.feedstock_directory)
  File "/Users/aaronmeurer/anaconda3/lib/python3.5/site-packages/conda_smithy/configure_feedstock.py", line 644, in main
    render_run_docker_build(env, config, forge_dir)
  File "/Users/aaronmeurer/anaconda3/lib/python3.5/site-packages/conda_smithy/configure_feedstock.py", line 77, in render_run_docker_build
    forge_config.get('channels', {}).get('sources', tuple())
  File "/Users/aaronmeurer/anaconda3/lib/python3.5/site-packages/conda_smithy/configure_feedstock.py", line 565, in compute_build_matrix
    mtx = special_case_version_matrix(meta, index)
  File "/Users/aaronmeurer/anaconda3/lib/python3.5/site-packages/conda_build_all/version_matrix.py", line 212, in special_case_version_matrix
    py_vn = minor_vn(index[python_pkg.fn]['version'])
KeyError: 'python-1.0.1-0.tar.bz2'

[rainwoodman]

We just looked at this. the newer version of conda uses python_pkg as the key of index. But it quickly gets more complicated and we gave up making a fix.

[asmeurer]

This seems like a conda bug. Why would the Resolve object return a package that isn't in the index? @kalefranz

[asmeurer]

Ping. This is making it impossible for me to rerender. CC @conda-forge/core @jakirkham @pelson

[jakirkham]

Are you using conda 4.2? If not, could you please try downgrading and retrying?

[asmeurer]

Seems to work after downgrading conda. I'd rather not be using an old version of conda, though.

[jakirkham]

Ok, well, I don't think anyone has had time to fix conda-build-all to work with conda 4.3. If you have time to investigate, that would definitely help make progress on this issue. @rainwoodman, do you remember what other issues/PRs are relevant for this?

Note: I'm not a maintainer of conda-build-all though have contributed a fix occasionally.

[asmeurer]

As I noted above, it smells like an issue in conda.

Side question: why is conda-build-all not a conda-forge project?

[rainwoodman]

Adding to the momentum: older version of conda-build is incompatible with miniconda3:

0.00s$ if [ $TRAVIS_OS_NAME == "osx" ]; then wget http://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -O miniconda.sh; fi

install.3

5.47s$ bash miniconda.sh -b -p $CONDA_INSTALL_LOCN

PREFIX=/home/travis/miniconda

installing: python-3.6.0-0 ...

installing: cffi-1.9.1-py36_0 ...

installing: conda-env-2.6.0-0 ...

installing: cryptography-1.7.1-py36_0 ...

installing: idna-2.2-py36_0 ...

installing: libffi-3.2.1-1 ...

installing: openssl-1.0.2k-0 ...

installing: pyasn1-0.1.9-py36_0 ...

installing: pycosat-0.6.1-py36_1 ...

installing: pycparser-2.17-py36_0 ...

installing: pyopenssl-16.2.0-py36_0 ...

installing: readline-6.2-2 ...

installing: requests-2.12.4-py36_0 ...

installing: ruamel_yaml-0.11.14-py36_1 ...

installing: setuptools-27.2.0-py36_0 ...

installing: six-1.10.0-py36_0 ...

installing: sqlite-3.13.0-0 ...

installing: tk-8.5.18-0 ...

installing: xz-5.2.2-1 ...

installing: yaml-0.1.6-0 ...

installing: zlib-1.2.8-3 ...

installing: conda-4.3.11-py36_0 ...

installing: pip-9.0.1-py36_1 ...

installing: wheel-0.29.0-py36_0 ...

Python 3.6.0 :: Continuum Analytics, Inc.

creating default environment...

installation finished.

install.4

0.00s$ export PATH=${CONDA_INSTALL_LOCN}/bin:$PATH

install.5

0.68s$ conda config --set always_yes true

install.6

0.00s$ PIN_FILE_CONDA=${CONDA_INSTALL_LOCN}/conda-meta/pinned

install.7

0.00s$ echo "conda ${CONDA_VERSION}" > $PIN_FILE_CONDA

install.8

7.19s$ conda update --quiet conda

Fetching package metadata .........

Solving package specifications: .

Package plan for installation in environment /home/travis/miniconda:

The following packages will be DOWNGRADED due to dependency conflicts:

    conda: 4.3.11-py36_0 --> 4.2.16-py36_0

install.9

32.30s$ conda install --quiet --yes astropy anaconda-client jinja2 cython pycrypto

Fetching package metadata .......

Solving package specifications: ..........

Package plan for installation in environment /home/travis/miniconda:

The following packages will be downloaded:

    package                    |            build

    ---------------------------|-----------------

    mkl-2017.0.1               |                0       128.2 MB

    cython-0.25.2              |           py36_0         6.5 MB

    markupsafe-0.23            |           py36_2          35 KB

    numpy-1.11.3               |           py36_0         6.7 MB

    pycrypto-2.6.1             |           py36_4         475 KB

    pytz-2016.10               |           py36_0         206 KB

    pyyaml-3.12                |           py36_0         345 KB

    astropy-1.3                |      np111py36_0         9.1 MB

    clyent-1.2.2               |           py36_0          18 KB

    jinja2-2.9.5               |           py36_0         389 KB

    python-dateutil-2.6.0      |           py36_0         237 KB

    anaconda-client-1.6.2      |           py36_0         130 KB

    ------------------------------------------------------------

                                           Total:       152.3 MB

The following NEW packages will be INSTALLED:

    anaconda-client: 1.6.2-py36_0   

    astropy:         1.3-np111py36_0

    clyent:          1.2.2-py36_0   

    cython:          0.25.2-py36_0  

    jinja2:          2.9.5-py36_0   

    markupsafe:      0.23-py36_2    

    mkl:             2017.0.1-0     

    numpy:           1.11.3-py36_0  

    pycrypto:        2.6.1-py36_4   

    python-dateutil: 2.6.0-py36_0   

    pytz:            2016.10-py36_0 

    pyyaml:          3.12-py36_0    

install.10

2.20s$ conda install ruamel_yaml

Fetching package metadata .......

Solving package specifications: ..........

# All requested packages already installed.

# packages in environment at /home/travis/miniconda:

#

ruamel_yaml               0.11.14                  py36_1  

install.11

18.13s$ conda install gcc

Fetching package metadata .......

Solving package specifications: ..........

Package plan for installation in environment /home/travis/miniconda:

The following packages will be downloaded:

    package                    |            build

    ---------------------------|-----------------

    gmp-6.1.0                  |                0         593 KB

    isl-0.12.2                 |                0         1.1 MB

    mpfr-3.1.5                 |                0         419 KB

    cloog-0.18.0               |                0         617 KB

    mpc-1.0.3                  |                0          61 KB

    gcc-4.8.5                  |                7        66.6 MB

    ------------------------------------------------------------

                                           Total:        69.4 MB

The following NEW packages will be INSTALLED:

    cloog: 0.18.0-0

    gcc:   4.8.5-7 

    gmp:   6.1.0-0 

    isl:   0.12.2-0

    mpc:   1.0.3-0 

    mpfr:  3.1.5-0 

Fetching packages ...

gmp-6.1.0-0.ta 100% || Time: 0:00:00  20.85 MB/s

isl-0.12.2-0.t 100% || Time: 0:00:00  25.49 MB/s

mpfr-3.1.5-0.t 100% || Time: 0:00:00  17.94 MB/s

cloog-0.18.0-0 100% || Time: 0:00:00  21.24 MB/s

mpc-1.0.3-0.ta 100% || Time: 0:00:00  19.92 MB/s

gcc-4.8.5-7.ta 100% || Time: 0:00:01  47.53 MB/s

Extracting packages ...

[      COMPLETE      ]|| 100%

Linking packages ...

No LSB modules are available.

[      COMPLETE      ]|| 100%

2.40s$ conda install conda-build=2.0.10

Fetching package metadata .......

Solving package specifications: ....

UnsatisfiableError: The following specifications were found to be in conflict:

  - conda-build 2.0.10*

  - python 3.6*

Use "conda info <package>" to see the dependencies for each package.

@asmeurer I disagree. This is not a conda-build issue. API of newer conda (4.3) is incompatible with 2.0.9 that conda-build-all depends on. The key of the index dictionary has changed, and using a string file name is no longer supported. Rather, there is now a data model for the packages with version components as attributes. The key of the index dictionary is now Package objects.

Conda-build-all does not use any of these. Thus a fix is going to be quite involved -- potentially a rewrite is needed. I think someone who actually wrote the old code (familiar with all of the heuristics) would be far more qualified to fix this; of course the assumption is the original author is still interested in this at all.

[jakirkham]

This should be fixed in the latest conda-build-all (1.0.4). Please try upgrading if you are still seeing this. If that still doesn't fix it, please let us know.

[rainwoodman]

I am curious -- which commit?

[jakirkham]

Ha! Yeah that's a good question. Things got a bit messy with the migration and a brief conda-forge interlude. Suffice it to say the fix is in there and we are testing against conda 4.3. So it should work.