PSLmodels / Package-Builder

Policy Simulation Library tool that builds packages and uploads them to the Anaconda Cloud's PSLmodels channel for public distribution
4 stars 7 forks source link

Allow for non-default channels such as conda-forge (try #2) #168

Closed hdoupe closed 5 years ago

hdoupe commented 5 years ago

Resolves #166. This PR updates the conda build string with the additional channels that are required by the package. They should be specified like this:

pbrelease Your-Package yourpackage 1.0.0 --channel conda-forge
hdoupe commented 5 years ago

Hmm, I'm still running into the same problem that @jdebacker described in #165. Here's the output when using commit https://github.com/PSLmodels/Cost-of-Capital-Calculator/commit/2e5289aee64085bb6a2ddaea46791ebf6f3b3f93:

(pkgbld-dev) hdoupe@pop-os:~/Documents/Cost-of-Capital-Calculator$ pbrelease Cost-of-Capital-Calculator ccc 0.0.0 --local --channel conda-forge
: Package-Builder will build model packages for:
:   repository_name = Cost-of-Capital-Calculator
:   package_name = ccc
:   model_version = 0.0.0
:   python_versions = ['3.7']
:   additional channels= ['conda-forge']
: Package-Builder will install package on local computer
: Package-Builder is starting at Mon May 20 14:56:53 2019
: Package-Builder is copying local source code
: Package-Builder is setting version
: Package-Builder is building package for Python 3.7
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
INFO:conda_build.variants:Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from config.variant
INFO:conda_build.metadata:Attempting to finalize metadata for ccc
WARNING: symlink_conda() is deprecated.
zip_safe flag not set; analyzing archive contents...
ccc.__pycache__.data.cpython-37: module references __file__
ccc.__pycache__.parameters.cpython-37: module references __file__
ccc.__pycache__.utils.cpython-37: module references __file__
INFO:conda_build.build:Packaging ccc
INFO:conda_build.build:Packaging ccc-0.0.0-py37_0
nm: dynamic_annotations.o: no symbols
Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
WARNING:conda_build.build:Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
INFO:conda_build.variants:Adding in variants from /tmp/tmprijn4i35/info/recipe/conda_build_config.yaml
WARNING: symlink_conda() is deprecated.
Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/test_tmp/run_test.py", line 2, in <module>
    import ccc
ModuleNotFoundError: No module named 'ccc'
Tests failed for ccc-0.0.0-py37_0.tar.bz2 - moving package to /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/broken
WARNING:conda_build.build:Tests failed for ccc-0.0.0-py37_0.tar.bz2 - moving package to /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/broken
TESTS FAILED: ccc-0.0.0-py37_0.tar.bz2
Traceback (most recent call last):
  File "/home/hdoupe/Documents/Package-Builder/pkgbld/utils.py", line 20, in os_call
    subprocess.run(cmd, shell=True, check=True, stdout=subprocess.PIPE)
  File "/home/hdoupe/anaconda3/envs/pkgbld-dev/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'conda build --python 3.7 --old-build-string  --channel conda-forge --channel pslmodels --override-channels --no-anaconda-upload --output-folder pkgbld_output conda.recipe' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-dev/bin/pbrelease", line 11, in <module>
    load_entry_point('pkgbld', 'console_scripts', 'pbrelease')()
  File "/home/hdoupe/Documents/Package-Builder/pkgbld/cli.py", line 109, in main
    channels=args.channel)
  File "/home/hdoupe/Documents/Package-Builder/pkgbld/release.py", line 199, in release
    u.os_call(cmd)
  File "/home/hdoupe/Documents/Package-Builder/pkgbld/utils.py", line 25, in os_call
    raise OSError(msg.format(err.returncode, cmd, err.output))
OSError: non-zero return code 1 generated by command:
conda build --python 3.7 --old-build-string  --channel conda-forge --channel pslmodels --override-channels --no-anaconda-upload --output-folder pkgbld_output conda.recipe
>output: b"Adding in variants from internal_defaults\nAdding in variants from config.variant\nAttempting to finalize metadata for ccc\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\nBUILD START: ['ccc-0.0.0-py37_0.tar.bz2']\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\n\n## Package Plan ##\n\n  environment location: /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p\n\n\nThe following NEW packages will be INSTALLED:\n\n    asn1crypto:      0.24.0-py37_1003      conda-forge\n    bokeh:           1.1.0-py37_0          conda-forge\n    bzip2:           1.0.6-h14c3975_1002   conda-forge\n    ca-certificates: 2019.3.9-hecc5488_0   conda-forge\n    certifi:         2019.3.9-py37_0       conda-forge\n    cffi:            1.12.3-py37h8022711_0 conda-forge\n    chardet:         3.0.4-py37_1003       conda-forge\n    cryptography:    2.6.1-py37h72c5cf5_0  conda-forge\n    freetype:        2.10.0-he983fc9_0     conda-forge\n    idna:            2.8-py37_1000         conda-forge\n    jinja2:          2.10.1-py_0           conda-forge\n    jpeg:            9c-h14c3975_1001      conda-forge\n    libblas:         3.8.0-10_openblas     conda-forge\n    libcblas:        3.8.0-10_openblas     conda-forge\n    libffi:          3.2.1-he1b5a44_1006   conda-forge\n    libgcc-ng:       8.2.0-hdf63c60_1                 \n    libgfortran-ng:  7.3.0-hdf63c60_0                 \n    liblapack:       3.8.0-10_openblas     conda-forge\n    libpng:          1.6.37-hed695b0_0     conda-forge\n    libstdcxx-ng:    8.2.0-hdf63c60_1                 \n    libtiff:         4.0.10-h648cc4a_1001  conda-forge\n    llvmlite:        0.28.0-py37hdbcaa40_0 conda-forge\n    markupsafe:      1.1.1-py37h14c3975_0  conda-forge\n    marshmallow:     3.0.0rc5-py_0         conda-forge\n    ncurses:         6.1-hf484d3e_1002     conda-forge\n    numba:           0.43.1-py37hf2d7682_0 conda-forge\n    numpy:           1.16.3-py37he5ce36f_0 conda-forge\n    olefile:         0.46-py_0             conda-forge\n    openblas:        0.3.6-h6e990d7_2      conda-forge\n    openssl:         1.1.1b-h14c3975_1     conda-forge\n    packaging:       19.0-py_0             conda-forge\n    pandas:          0.24.2-py37hf484d3e_0 conda-forge\n    paramtools:      0.5.4-py_0            conda-forge\n    pillow:          6.0.0-py37he7afcd5_0  conda-forge\n    pip:             19.1-py37_0           conda-forge\n    pycparser:       2.19-py37_1           conda-forge\n    pyopenssl:       19.0.0-py37_0         conda-forge\n    pyparsing:       2.4.0-py_0            conda-forge\n    pysocks:         1.7.0-py37_0          conda-forge\n    python:          3.7.3-h5b0a415_0      conda-forge\n    python-dateutil: 2.8.0-py_0            conda-forge\n    pytz:            2019.1-py_0           conda-forge\n    pyyaml:          5.1-py37h14c3975_0    conda-forge\n    readline:        7.0-hf8c457e_1001     conda-forge\n    requests:        2.22.0-py37_0         conda-forge\n    scipy:           1.3.0-py37hab63836_0  conda-forge\n    setuptools:      41.0.1-py37_0         conda-forge\n    six:             1.12.0-py37_1000      conda-forge\n    sqlite:          3.26.0-h67949de_1001  conda-forge\n    taxcalc:         2.2.0-py37_0          pslmodels  \n    tk:              8.6.9-h84994c4_1001   conda-forge\n    tornado:         6.0.2-py37h516909a_0  conda-forge\n    urllib3:         1.24.3-py37_0         conda-forge\n    wheel:           0.33.4-py37_0         conda-forge\n    xlrd:            1.2.0-py_0            conda-forge\n    xz:              5.2.4-h14c3975_1001   conda-forge\n    yaml:            0.1.7-h14c3975_1001   conda-forge\n    zlib:            1.2.11-h14c3975_1004  conda-forge\n\nPreparing transaction: ...working... done\nVerifying transaction: ...working... done\nExecuting transaction: ...working... done\nCopying /home/hdoupe/temporary_pkgbld_working_dir/Cost-of-Capital-Calculator to /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work\nsource tree in: /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work\nexport PREFIX=/home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p\nexport SRC_DIR=/home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work\n~/temporary_pkgbld_working_dir/Cost-of-Capital-Calculator ~/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work\nrunning install\nrunning bdist_egg\nrunning egg_info\nwriting ccc.egg-info/PKG-INFO\nwriting dependency_links to ccc.egg-info/dependency_links.txt\nwriting requirements to ccc.egg-info/requires.txt\nwriting top-level names to ccc.egg-info/top_level.txt\nreading manifest file 'ccc.egg-info/SOURCES.txt'\nreading manifest template 'MANIFEST.in'\nwriting manifest file 'ccc.egg-info/SOURCES.txt'\ninstalling library code to build/bdist.linux-x86_64/egg\nrunning install_lib\nrunning build_py\ncopying ccc/__init__.py -> build/lib/ccc\ncreating build/bdist.linux-x86_64/egg\ncreating build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/get_taxcalc_rates.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/styles.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/ccc_asset_data.csv -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/calcfunctions.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/records_variables.json -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/__init__.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/calculator.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/constants.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/default_parameters.json -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/utils.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/data.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/controls_callback_script.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/parameters.py -> build/bdist.linux-x86_64/egg/ccc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/get_taxcalc_rates.py to get_taxcalc_rates.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/styles.py to styles.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/calcfunctions.py to calcfunctions.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/__init__.py to __init__.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/calculator.py to calculator.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/constants.py to constants.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/utils.py to utils.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/data.py to data.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/controls_callback_script.py to controls_callback_script.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/parameters.py to parameters.cpython-37.pyc\ncreating build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncreating 'dist/ccc-0.0.0-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it\nremoving 'build/bdist.linux-x86_64/egg' (and everything under it)\nProcessing ccc-0.0.0-py3.7.egg\ncreating $PREFIX/lib/python3.7/site-packages/ccc-0.0.0-py3.7.egg\nExtracting ccc-0.0.0-py3.7.egg to $PREFIX/lib/python3.7/site-packages\nAdding ccc 0.0.0 to easy-install.pth file\n\nInstalled $PREFIX/lib/python3.7/site-packages/ccc-0.0.0-py3.7.egg\nProcessing dependencies for ccc==0.0.0\nSearching for xlrd==1.2.0\nBest match: xlrd 1.2.0\nAdding xlrd 1.2.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for scipy==1.3.0\nBest match: scipy 1.3.0\nAdding scipy 1.3.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for taxcalc==2.2.0\nBest match: taxcalc 2.2.0\nAdding taxcalc 2.2.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for pandas==0.24.2\nBest match: pandas 0.24.2\nAdding pandas 0.24.2 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for numpy==1.16.3\nBest match: numpy 1.16.3\nAdding numpy 1.16.3 to easy-install.pth file\nInstalling f2py script to $PREFIX/bin\nInstalling f2py3 script to $PREFIX/bin\nInstalling f2py3.7 script to $PREFIX/bin\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for numba==0.43.1\nBest match: numba 0.43.1\nAdding numba 0.43.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for bokeh==1.1.0\nBest match: bokeh 1.1.0\nAdding bokeh 1.1.0 to easy-install.pth file\nInstalling bokeh script to $PREFIX/bin\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for pytz==2019.1\nBest match: pytz 2019.1\nAdding pytz 2019.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for python-dateutil==2.8.0\nBest match: python-dateutil 2.8.0\nAdding python-dateutil 2.8.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for llvmlite==0.28.0\nBest match: llvmlite 0.28.0\nAdding llvmlite 0.28.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for packaging==19.0\nBest match: packaging 19.0\nAdding packaging 19.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for tornado==6.0.2\nBest match: tornado 6.0.2\nAdding tornado 6.0.2 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for Jinja2==2.10.1\nBest match: Jinja2 2.10.1\nAdding Jinja2 2.10.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for PyYAML==5.1\nBest match: PyYAML 5.1\nAdding PyYAML 5.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for Pillow==6.0.0\nBest match: Pillow 6.0.0\nAdding Pillow 6.0.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for six==1.12.0\nBest match: six 1.12.0\nAdding six 1.12.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for pyparsing==2.4.0\nBest match: pyparsing 2.4.0\nAdding pyparsing 2.4.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for MarkupSafe==1.1.1\nBest match: MarkupSafe 1.1.1\nAdding MarkupSafe 1.1.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nFinished processing dependencies for ccc==0.0.0\n~/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work\n\nResource usage statistics from building ccc:\n   Process count: 1\n   CPU time: unavailable\n   Memory: 3.3M\n   Disk usage: 193.5K\n   Time elapsed: 0:00:02.1\n\nPackaging ccc\nPackaging ccc-0.0.0-py37_0\ncompiling .pyc files...\nfound egg dir: /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/ccc-0.0.0-py3.7.egg\nnumber of files: 29\n   INFO (ccc): plugin library package conda-forge::scipy-1.3.0-py37hab63836_0 in requirements/run but it is not used (i.e. it is overdepending or perhaps statically linked? If that is what you want then add it to `build/ignore_run_exports`)\nFixing permissions\nCompressing to /tmp/tmpl93uxrz6/ccc-0.0.0-py37_0.tar.bz2\nTEST START: pkgbld_output/linux-64/ccc-0.0.0-py37_0.tar.bz2\nAdding in variants from /tmp/tmprijn4i35/info/recipe/conda_build_config.yaml\nRenaming work directory,  /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work  to  /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/work_moved_ccc-0.0.0-py37_0_linux-64\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\n\n## Package Plan ##\n\n  environment location: /home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol\n\n\nThe following NEW packages will be INSTALLED:\n\n    asn1crypto:      0.24.0-py36_1003      conda-forge                                                                              \n    bokeh:           1.1.0-py36_0          conda-forge                                                                              \n    ca-certificates: 2019.3.9-hecc5488_0   conda-forge                                                                              \n    ccc:             0.0.0-py37_0          file:///home/hdoupe/temporary_pkgbld_working_dir/Cost-of-Capital-Calculator/pkgbld_output\n    certifi:         2019.3.9-py36_0       conda-forge                                                                              \n    cffi:            1.12.3-py36h8022711_0 conda-forge                                                                              \n    chardet:         3.0.4-py36_1003       conda-forge                                                                              \n    cryptography:    2.6.1-py36h72c5cf5_0  conda-forge                                                                              \n    freetype:        2.10.0-he983fc9_0     conda-forge                                                                              \n    idna:            2.8-py36_1000         conda-forge                                                                              \n    jinja2:          2.10.1-py_0           conda-forge                                                                              \n    jpeg:            9c-h14c3975_1001      conda-forge                                                                              \n    libblas:         3.8.0-10_openblas     conda-forge                                                                              \n    libcblas:        3.8.0-10_openblas     conda-forge                                                                              \n    libffi:          3.2.1-he1b5a44_1006   conda-forge                                                                              \n    libgcc-ng:       8.2.0-hdf63c60_1                                                                                               \n    libgfortran-ng:  7.3.0-hdf63c60_0                                                                                               \n    liblapack:       3.8.0-10_openblas     conda-forge                                                                              \n    libpng:          1.6.37-hed695b0_0     conda-forge                                                                              \n    libstdcxx-ng:    8.2.0-hdf63c60_1                                                                                               \n    libtiff:         4.0.10-h648cc4a_1001  conda-forge                                                                              \n    llvmlite:        0.28.0-py36hdbcaa40_0 conda-forge                                                                              \n    markupsafe:      1.1.1-py36h14c3975_0  conda-forge                                                                              \n    marshmallow:     3.0.0rc5-py_0         conda-forge                                                                              \n    ncurses:         6.1-hf484d3e_1002     conda-forge                                                                              \n    numba:           0.43.1-py36hf2d7682_0 conda-forge                                                                              \n    numpy:           1.16.3-py36he5ce36f_0 conda-forge                                                                              \n    olefile:         0.46-py_0             conda-forge                                                                              \n    openblas:        0.3.6-h6e990d7_2      conda-forge                                                                              \n    openssl:         1.1.1b-h14c3975_1     conda-forge                                                                              \n    packaging:       19.0-py_0             conda-forge                                                                              \n    pandas:          0.24.2-py36hf484d3e_0 conda-forge                                                                              \n    paramtools:      0.5.4-py_0            conda-forge                                                                              \n    pillow:          6.0.0-py36he7afcd5_0  conda-forge                                                                              \n    pip:             19.1-py36_0           conda-forge                                                                              \n    pycparser:       2.19-py36_1           conda-forge                                                                              \n    pyopenssl:       19.0.0-py36_0         conda-forge                                                                              \n    pyparsing:       2.4.0-py_0            conda-forge                                                                              \n    pysocks:         1.7.0-py36_0          conda-forge                                                                              \n    python:          3.6.7-h381d211_1004   conda-forge                                                                              \n    python-dateutil: 2.8.0-py_0            conda-forge                                                                              \n    pytz:            2019.1-py_0           conda-forge                                                                              \n    pyyaml:          5.1-py36h14c3975_0    conda-forge                                                                              \n    readline:        7.0-hf8c457e_1001     conda-forge                                                                              \n    requests:        2.22.0-py36_0         conda-forge                                                                              \n    scipy:           1.3.0-py36hab63836_0  conda-forge                                                                              \n    setuptools:      41.0.1-py36_0         conda-forge                                                                              \n    six:             1.12.0-py36_1000      conda-forge                                                                              \n    sqlite:          3.26.0-h67949de_1001  conda-forge                                                                              \n    taxcalc:         2.2.0-py36_0          pslmodels                                                                                \n    tk:              8.6.9-h84994c4_1001   conda-forge                                                                              \n    tornado:         6.0.2-py36h516909a_0  conda-forge                                                                              \n    urllib3:         1.24.3-py36_0         conda-forge                                                                              \n    wheel:           0.33.4-py36_0         conda-forge                                                                              \n    xlrd:            1.2.0-py_0            conda-forge                                                                              \n    xz:              5.2.4-h14c3975_1001   conda-forge                                                                              \n    yaml:            0.1.7-h14c3975_1001   conda-forge                                                                              \n    zlib:            1.2.11-h14c3975_1004  conda-forge                                                                              \n\nPreparing transaction: ...working... done\nVerifying transaction: ...working... done\nExecuting transaction: ...working... done\nexport PREFIX=/home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol\nexport SRC_DIR=/home/hdoupe/anaconda3/envs/pkgbld-dev/conda-bld/ccc_1558378614407/test_tmp\nimport: 'ccc'\n"
hdoupe commented 5 years ago

Another approach to this issue is to use a less complicated, more streamlined approach with the noarch option. This approach was used to successfully package and upload CCC in this PR: https://github.com/PSLmodels/Cost-of-Capital-Calculator/pull/261. I mistakenly thought that noarch meant all platforms except for "arch linux". However, it actually means "no architecture." The implication is that a package is built once and works for all platforms and python versions. It looks like the only requirement is that the package must be pure python, i.e. without any C extensions. This is also the recommended build configuration by conda-forge.

If we go this route, then projects would need to make a couple modifications to their meta.yaml files (along the lines of the changes here). Projects would also no longer need the build.sh or bld.bat scripts.

I made some modifications to Package-Builder so that it would use this approach. You can check those out on my use-noarch branch. I was able to locally build CCC on this commit https://github.com/PSLmodels/Cost-of-Capital-Calculator/commit/d017fb2b994479e23d86f4f5f263f2872b075e92.

I'm curious to hear what people think about this approach.

cc @jdebacker @martinholmer

jdebacker commented 5 years ago

@hdoupe I don't know any specifics about noarch, but I like the effort to allow other packages from other channels in Package Builder.

martinholmer commented 5 years ago

@hdoupe said in Package-Builder pull request #168:

Another approach to this issue is to use a less complicated, more streamlined approach with the noarch option. This approach was used to successfully package and upload CCC in this PR: PSLmodels/Cost-of-Capital-Calculator#261. I mistakenly thought that noarch meant all platforms except for "arch linux". However, it actually means "no architecture." The implication is that a package is built once and works for all platforms and python versions. It looks like the only requirement is that the package must be pure python, i.e. without any C extensions. This is also the recommended build configuration by conda-forge.

If we go this route, then projects would need to make a couple modifications to their meta.yaml files (along the lines of the changes here). Projects would also no longer need the build.sh or bld.bat scripts.

I made some modifications to Package-Builder so that it would use this approach. You can check those out on my use-noarch branch. I was able to locally build CCC on this commit PSLmodels/Cost-of-Capital-Calculator@d017fb2.

I'm curious to hear what people think about this approach.

cc @jdebacker @martinholmer

I think this approach is very bad from the perspective of the package users. I think that for two reasons: one is a minor problem and the other is a major problem.

First the minor problem with this approach. The documentation of Tax-Calculator explicitly tells users all they need to do in install Anaconda Python and download the model package. And I think all the other models in the Policy Simulation Library, including Cost-of-Capital-Calculator, suggest (at least implicitly) the same thing. So, currently users are being asked to install the models with this command:

conda install -c PSLmodels ccc

So, the change you suggest might require rewriting model documentation and would definitely require telling users to use this longer command:

conda install -c conda-forge -c PSLmodels ccc

Not only is this more typing, but it likely raises in the user's mind a question like: "What in the world is conda-forge? I though I was installing a PSL model." This is a marketing/branding problem. Perhaps minor, but real.

Next let's look at the far bigger problem with your suggested approach. I'm puzzled why you didn't mention this in your request for reactions to your suggested approach. Right now the conda package installation times are barely acceptable, but if we follow your advice, the package install times will increase five-fold making the process of installing the conda package for a PSL model unacceptably long.

Let me provides some details on the experiments I've run on my computer. First, let me provide some context. I know from working on the phone with Jason DeBacker, who has a Mac notebook, and Cody Kallen, who has a Windows notebook, that my Mac desktop is substantially faster than either of those two computers. And I know that many users are using notebooks like Jason and Cody. The other bit of context is that on my previous project (a pension and social security dynamic microsimulation model), the whole installation package, which included three multi-megabyte executables, several large SQLite database files, hundreds of large HTML files, and several large PDF files (containing documents that are hundreds of pages long), downloaded from the web and was installed on the user's Windows computer in a few seconds.

Let's compare the installation times on my computer for the Cost-of-Capital-Calculator ccc package (which is only 2.1 megabytes in size), remembering that these times are almost certainly longer on a typical user's computer.

That is slightly more the four and a half minutes of watching the conda install spinner spin around and around. Given my experience this is unacceptably long. I'm a user of these packages and I don't want to be waiting that long for a model install, something that I do often during the course of a day of model development work.

MattHJensen commented 5 years ago

I'd be interested to hear Hank's reply to your comments, @martinholmer. Perhaps he could address your concerns through further commits to the PR, or perhaps he has a different view about priorities and would like to convey that view to the community. Could you please reopen this PR?

MattHJensen commented 5 years ago

Purely from a philosophical standpoint, I tend to think that project maintainers using Package-Builder should be trusted to make decisions about tradeoffs like the number of install lines or the installation time.

martinholmer commented 5 years ago

I'm reopening Package-Builder pull request #168 at Matt Jensen's request even thought the person submitting the pull request admitted it doesn't work. That is, it doesn't make a package at all; not that the package install is slow.

martinholmer commented 5 years ago

@MattHJensen said in Package-Builder pull request #168:

Purely from a philosophical standpoint, I tend to think that project maintainers using Package-Builder should be trusted to make decisions about tradeoffs like the number of install lines or the installation time.

I would agree that "project maintainers should be trusted to make decisions about tradeoffs like the number of install lines or the installation time." If project maintainers want to use the slow technology, then they can create a repository (maybe called something like Slow-Package-Builder) and use that tool to make conda packages for their models. I've got no problem with that approach. The problem I would have is if other people forced a change in Package-Builder, which is working just fine making packages for Tax-Calculator 2.2.0, Behavioral-Responses 0.8.0, Tax-Brain 2.2.0, and Cost-of-Capital-Calculator 1.0.1.

martinholmer commented 5 years ago

@hdoupe, I want to make sure I understand the state of play with regards to your work in PR #167 and #168.

I understood you to mean in this comment that you added a --channel option to Package-Builder so that it would look for packages in channels other than the Anaconda defaults channels and the PSLmodels channel, but this enhanced version of Package-Builder still ran into the same problem as @jdebacker reported in issue #165.

Is that correct?

If so, it suggests to me that the problem reported by Jason is not rooted in the channel location of the paramtools package, but rather is rooted in the intrinsic nature of the paramtools package. Am I missing other possible source's of Jason's problem? If his problem is rooted in the intrinsic nature of the most recent paramtools package, what do you think could be the problem? If there are other possibilities that are causing Jason's problem, please explain those other possibilities to us.

hdoupe commented 5 years ago

I think it would be helpful for me to state where I'm coming from with PR #168 and issue #166. First, here is the problem: Projects that depend on on ParamTools can no longer use Package-Builder.

There is no inherent problem with ParamTools: it is set up just like every other package in the PSLmodels ecosystem, except that it's on the conda-forge channel. It's on the conda-forge channel because it depends on Marshmallow, a widely used serialization package with 3,600 stars on GitHub and an active community that was friendly enough to respond to my comments and even merge a PR that I opened. Marshmallow is only available on the conda-forge channel and not on the defaults channel. ParamTools will be used by projects both inside and outside of the PSLmodels community, and thus, it made sense to package it on conda-forge. That way, users who are not familiar with PSL can just install it like conda install -c conda-forge paramtools without having to know about the PSLmodels channel.

Asking users to specify -c conda-forge when they install a PSLmodels package does not seem like much of an issue. One reason is that the documentation could be updated to something like this:

When first installing PSLmodels packages, you need to add two channels to your default list of channels:

conda config add --channels pslmodels
conda config add --channels conda-forge

Now, you can install Tax-Calculator with this command: conda install taxcalc and you can update your existing installation with this command: conda update taxcalc.

I think this is fine. The other reason is that conda-forge is a really cool project. The on-boarding process is super smooth. I published ParamTools on that channel in a single afternoon and have not had a problem since. Conda-forge is also a NumFocus sponsored, community maintained project. It allows packages to easily publish on a well-known conda channel without having to persuade the Anaconda corporation to do it a favor and publish it in the defaults channel. So, if this is extra marketing for the conda-forge project, then I'm all for it.

With this knowledge, it seems like the best course of action is to enhance Package-Builder so that it can build packages with dependencies that are not in the defaults or PSLmodels conda channel. I was unsuccessful when I initially tried it after @jdebacker first reported this problem. Next, I tried to take a fresh approach to this problem and successfully packaged CCC on the noarch platform which is the default and recommended way to package pure-python software on conda-forge. Further, after a few minor changes to the meta.yaml file, it only took 3 bash commands to publish it!

export CONDA_TOKEN=your_anaconda_token
conda config --set anaconda_upload yes
conda build conda.recipe/ --token $CONDA_TOKEN --user pslmodels

This seems like a vast improvement over the current system which requires (Number Python Versions * Number of OS platform) builds. Not only is it more efficient, it would simplify the Package-Builder codebase meaning that it would require less maintenance and hopefully, "just work."

In this comment @martinholmer mentions another kind of efficiency:

Let's compare the installation times on my computer for the Cost-of-Capital-Calculator ccc package (which is only 2.1 megabytes in size), remembering that these times are almost certainly longer on a typical user's computer.

  • ccc 1.0.1 package (which uses the current Package-Builder technology) installs in 54 seconds

  • ccc 1.0.0 package (which uses your suggested technology) installs in 277 seconds

I understand that the install time is slow. However, I suspect that users will install a package once and only update it when releases come out. This works out to something like re-installing packages once every week or every other week. So, the time cost for users is 2 or 3 minutes a week at the worst. This is a small price to pay for a Package-Builder that is easier to understand and maintain, is more reliable, and supports packages on more conda channels.

martinholmer commented 5 years ago

@hdoupe, Thanks for your lengthy comment, but I don't believe you answered the questions I posed in my prior comment. The main question I have is this:

Why is it that when you enhanced Package-Builder to look on the conda-forge channel in PR #168 that you could not solve @jdebacker's problem?

hdoupe commented 5 years ago

@martinholmer asked:

Why is it that when you enhanced Package-Builder to look on the conda-forge channel in PR #168 that you could not solve @jdebacker's problem?

I have no idea. From looking through the error log, we can see that ParamTools is installed. So, it seems like specifying the channel allowed all of the dependencies, including ParamTools, to be installed. It's a bit of a mystery why CCC could not be imported after that though. @martinholmer do you have any ideas for what might be happening?

martinholmer commented 5 years ago

@hdoupe said in Package-Builder PR #168:

Why is it that when you enhanced Package-Builder to look on the conda-forge channel in PR #168 that you could not solve @jdebacker's problem?

I have no idea. From looking through the error log, we can see that ParamTools is installed. So, it seems like specifying the channel allowed all of the dependencies, including ParamTools, to be installed. It's a bit of a mystery why CCC could not be imported after that though. @martinholmer do you have any ideas for what might be happening?

I don't know for sure what is happening. But it would seem as if something has changed in ParmaTools since mid-March. It was then that it appears as if you used Package-Builder to build and upload to the PSLmodels channel eight (architecture-specific) packages for ParamTools release 0.4.1. Here is what's still up on the PSLmodels channel:

Screen Shot 2019-05-23 at 12 45 21 PM

Other than taking a long time to install (because marshmallow is a noarch package), the paramtools packages download and seem to work fine. Here is my experience:

iMac:~ mrh$ conda search -c PSLmodels paramtools
Loading channels: done
# Name                       Version           Build  Channel             
paramtools                     0.4.0          py36_0  PSLmodels           
paramtools                     0.4.0          py37_0  PSLmodels           
paramtools                     0.4.1          py36_0  PSLmodels           
paramtools                     0.4.1          py37_0  PSLmodels      

iMac:~ mrh$ conda install -c PSLmodels paramtools=0.4.1
Collecting package metadata: done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - paramtools=0.4.1 -> marshmallow[version='>=3.0.0rc4']

Current channels:

  - https://conda.anaconda.org/PSLmodels/osx-64
  - https://conda.anaconda.org/PSLmodels/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/osx-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

iMac:~ mrh$ conda install -c conda-forge -c PSLmodels paramtools=0.4.1
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /Users/mrh/anaconda3

  added / updated specs:
    - paramtools=0.4.1

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2019.3.9           |           py36_0         149 KB  conda-forge
    conda-4.6.14               |           py36_0         2.1 MB  conda-forge
    paramtools-0.4.1           |             py_0          44 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         2.3 MB

The following NEW packages will be INSTALLED:

  marshmallow        conda-forge/noarch::marshmallow-3.0.0rc5-py_0
  paramtools         conda-forge/noarch::paramtools-0.4.1-py_0

The following packages will be UPDATED:

  ca-certificates    pkgs/main::ca-certificates-2019.1.23-0 --> conda-forge::ca-certificates-2019.3.9-hecc5488_0
  openssl              pkgs/main::openssl-1.1.1b-h1de35cc_1 --> conda-forge::openssl-1.1.1b-h01d97ff_2

The following packages will be SUPERSEDED by a higher-priority channel:

  certifi                                         pkgs/main --> conda-forge
  conda                                           pkgs/main --> conda-forge

Proceed ([y]/n)? y

Downloading and Extracting Packages
conda-4.6.14         | 2.1 MB    | ##################################### | 100% 
paramtools-0.4.1     | 44 KB     | ##################################### | 100% 
certifi-2019.3.9     | 149 KB    | ##################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

iMac:~ mrh$ python 
Python 3.6.8 |Anaconda custom (64-bit)| (default, Dec 29 2018, 19:04:46) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import paramtools
>>> print(paramtools.__version__)
0.4.1
>>> exit(0)

So it looks like you were successful in building paramtool packages in March with Package-Builder. Or did you build and upload the 0.4.1 packages with some other tool?

Can you still build and upload to the PSLmodels channel, architecture-specific paramtools packages for ParamTools release 0.5.4? If so, will using those packages allow you to build ccc 1.0.0 packages with the PR#168-enhanced Package-Builder?

By the way, why did you stop building paramtools packages for the PSLmodels channel?

hdoupe commented 5 years ago

@martinholmer asked:

So it looks like you were successful in building paramtool packages in March with Package-Builder. Or did you build and upload the 0.4.1 packages with some other tool?

Correct, I built and published ParamTools 0.4.1 with Package-Builder.

Can you still build and upload to the PSLmodels channel, architecture-specific paramtools packages for ParamTools release 0.5.4? If so, will using those packages allow you to build ccc 1.0.0 packages with the PR#168-enhanced Package-Builder?

I think so. I'll give that a shot.

By the way, why did you stop building paramtools packages for the PSLmodels channel?

I didn't see an upside to publishing it on the PSLmodels channel in addition to the conda-forge channel because I didn't think users would want to install a package like this conda install -c conda-forge -c pslmodels paramtools when they could just use conda install -c conda-forge paramtools.

hdoupe commented 5 years ago

I uploaded ParamTools 0.5.4 to the PSLmodels channel with Package-Builder 0.22.0 without any problems. However, I got the same error that we've been getting when packaging and building CCC:

(pkgbld-env) hdoupe@pop-os:~$ pbrelease ParamTools paramtools 0.5.4 
: Package-Builder will build model packages for:
:   repository_name = ParamTools
:   package_name = paramtools
:   model_version = 0.5.4
:   python_versions = ['3.7', '3.6']
: Package-Builder will upload model packages to:
:   Anaconda channel = pslmodels
:   using token in file = /home/hdoupe/.pslmodels_anaconda_token
: Package-Builder is starting at Thu May 23 13:27:05 2019
: Package-Builder is cloning repository code for 0.5.4
Cloning into 'ParamTools'...
remote: Enumerating objects: 54, done.
remote: Counting objects: 100% (54/54), done.
remote: Compressing objects: 100% (47/47), done.
remote: Total 54 (delta 2), reused 22 (delta 0), pack-reused 0
Unpacking objects: 100% (54/54), done.
Note: checking out '41c0bce3afd7d834b57d2d4245840c489bec4c39'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

: Package-Builder is setting version
: Package-Builder is building package for Python 3.7
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
INFO:conda_build.variants:Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from config.variant
INFO:conda_build.metadata:Attempting to finalize metadata for paramtools
WARNING: symlink_conda() is deprecated.
zip_safe flag not set; analyzing archive contents...
paramtools.__pycache__.utils.cpython-37: module references __file__
paramtools.tests.__pycache__.test_parameters.cpython-37: module references __file__
paramtools.tests.test_examples.__pycache__.test_baseball.cpython-37: module references __file__
paramtools.tests.test_examples.__pycache__.test_behresp.cpython-37: module references __file__
paramtools.tests.test_examples.__pycache__.test_tc_ex.cpython-37: module references __file__
INFO:conda_build.build:Packaging paramtools
INFO:conda_build.build:Packaging paramtools-0.5.4-py37_0
nm: dynamic_annotations.o: no symbols
Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
WARNING:conda_build.build:Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
INFO:conda_build.variants:Adding in variants from /tmp/tmpc0bq623q/info/recipe/conda_build_config.yaml
WARNING: symlink_conda() is deprecated.
: Package-Builder is converting package for Python 3.7
: Package-Builder is uploading packages for Python 3.7
Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/osx-64/paramtools-0.5.4-py37_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/linux-64/paramtools-0.5.4-py37_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/win-32/paramtools-0.5.4-py37_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/win-64/paramtools-0.5.4-py37_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

: Package-Builder is building package for Python 3.6
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
INFO:conda_build.variants:Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from config.variant
INFO:conda_build.metadata:Attempting to finalize metadata for paramtools
WARNING: symlink_conda() is deprecated.
zip_safe flag not set; analyzing archive contents...
paramtools.__pycache__.utils.cpython-36: module references __file__
paramtools.tests.__pycache__.test_parameters.cpython-36: module references __file__
paramtools.tests.test_examples.__pycache__.test_baseball.cpython-36: module references __file__
paramtools.tests.test_examples.__pycache__.test_behresp.cpython-36: module references __file__
paramtools.tests.test_examples.__pycache__.test_tc_ex.cpython-36: module references __file__
INFO:conda_build.build:Packaging paramtools
INFO:conda_build.build:Packaging paramtools-0.5.4-py36_0
nm: dynamic_annotations.o: no symbols
Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
WARNING:conda_build.build:Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
INFO:conda_build.variants:Adding in variants from /tmp/tmpg02x0_nj/info/recipe/conda_build_config.yaml
WARNING: symlink_conda() is deprecated.
: Package-Builder is converting package for Python 3.6
: Package-Builder is uploading packages for Python 3.6
Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/osx-64/paramtools-0.5.4-py36_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/linux-64/paramtools-0.5.4-py36_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/win-32/paramtools-0.5.4-py36_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

Using Anaconda API: https://api.anaconda.org
Using "pslmodels" as upload username
Detecting file type...
File type is "conda"
Extracting package attributes for upload
Creating package "paramtools"
Creating release "0.5.4"
Uploading file "pslmodels/paramtools/0.5.4/win-64/paramtools-0.5.4-py36_0.tar.bz2"
 uploaded 46 of 46Kb: 100.00% ETA: 0.0 minutes
Upload complete
package located at:
https://anaconda.org/pslmodels/paramtools

: Package-Builder is cleaning-up
: Package-Builder is finishing at Thu May 23 13:36:05 2019
(pkgbld-env) hdoupe@pop-os:~$ pbrelease Cost-of-Capital-Calculator ccc 1.0.0
: Package-Builder will build model packages for:
:   repository_name = Cost-of-Capital-Calculator
:   package_name = ccc
:   model_version = 1.0.0
:   python_versions = ['3.7', '3.6']
: Package-Builder will upload model packages to:
:   Anaconda channel = pslmodels
:   using token in file = /home/hdoupe/.pslmodels_anaconda_token
: Package-Builder is starting at Thu May 23 13:38:46 2019
: Package-Builder is cloning repository code for 1.0.0
Cloning into 'Cost-of-Capital-Calculator'...
remote: Enumerating objects: 135, done.
remote: Counting objects: 100% (135/135), done.
remote: Compressing objects: 100% (126/126), done.
remote: Total 135 (delta 7), reused 68 (delta 4), pack-reused 0
Receiving objects: 100% (135/135), 31.97 MiB | 8.83 MiB/s, done.
Resolving deltas: 100% (7/7), done.
Note: checking out '2e5289aee64085bb6a2ddaea46791ebf6f3b3f93'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

: Package-Builder is setting version
: Package-Builder is building package for Python 3.7
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
INFO:conda_build.variants:Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from config.variant
INFO:conda_build.metadata:Attempting to finalize metadata for ccc
Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 456, in main
    execute(sys.argv[1:])
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 447, in execute
    verify=args.verify, variants=args.variants)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/api.py", line 208, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/build.py", line 2314, in build_tree
    notest=notest,
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/build.py", line 1324, in build
    output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/render.py", line 735, in expand_outputs
    for (output_dict, m) in _m.copy().get_output_metadata_set(permit_unsatisfiable_variants=False):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/metadata.py", line 2038, in get_output_metadata_set
    bypass_env_check=bypass_env_check)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/metadata.py", line 721, in finalize_outputs_pass
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/render.py", line 509, in finalize_metadata
    exclude_pattern)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/render.py", line 371, in add_upstream_pins
    permit_unsatisfiable_variants, exclude_pattern)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/render.py", line 362, in _read_upstream_pin_files
    extra_run_specs = get_upstream_pins(m, actions, env)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/render.py", line 344, in get_upstream_pins
    package_subset=linked_packages)
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda_build/render.py", line 323, in execute_download_actions
    pfe.execute()
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/conda/core/package_cache_data.py", line 620, in execute
    raise CondaMultiError(exceptions)
conda.CondaMultiError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/pslmodels/linux-64/taxcalc-2.2.0-py37_0.tar.bz2>
Elapsed: -

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/utils.py", line 20, in os_call
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'conda build --python 3.7 --old-build-string --channel pslmodels --override-channels --no-anaconda-upload --output-folder pkgbld_output conda.recipe' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/bin/pbrelease", line 11, in <module>
    sys.exit(main())
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/cli.py", line 103, in main
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/release.py", line 189, in release
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/utils.py", line 25, in os_call
OSError: non-zero return code 1 generated by command:
conda build --python 3.7 --old-build-string --channel pslmodels --override-channels --no-anaconda-upload --output-folder pkgbld_output conda.recipe
>output: b'Adding in variants from internal_defaults\nAdding in variants from config.variant\nAttempting to finalize metadata for ccc\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\n'
(pkgbld-env) hdoupe@pop-os:~$ pbrelease --version
Package-Builder 0.22.0
(pkgbld-env) hdoupe@pop-os:~$ 
martinholmer commented 5 years ago

@hdoupe, Thnaks for trying these thing out. The build of the CCC 1.0.0 packages seems to fail with this msg:

conda.CondaMultiError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/pslmodels/linux-64/taxcalc-2.2.0-py37_0.tar.bz2>
Elapsed: -

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

What do suppose that is all about? Do you have problems on your Linux computer installing taxcalc 2.2.0?

hdoupe commented 5 years ago

Whoops, I was having some wifi issues. Here's the output:

(pkgbld-env) hdoupe@pop-os:~$ conda build purge
(pkgbld-env) hdoupe@pop-os:~$ pbrelease Cost-of-Capital-Calculator ccc 1.0.0
: Package-Builder will build model packages for:
:   repository_name = Cost-of-Capital-Calculator
:   package_name = ccc
:   model_version = 1.0.0
:   python_versions = ['3.7', '3.6']
: Package-Builder will upload model packages to:
:   Anaconda channel = pslmodels
:   using token in file = /home/hdoupe/.pslmodels_anaconda_token
: Package-Builder is starting at Thu May 23 14:58:22 2019
: Package-Builder is cloning repository code for 1.0.0
Cloning into 'Cost-of-Capital-Calculator'...
remote: Enumerating objects: 135, done.
remote: Counting objects: 100% (135/135), done.
remote: Compressing objects: 100% (126/126), done.
remote: Total 135 (delta 7), reused 68 (delta 4), pack-reused 0
Receiving objects: 100% (135/135), 31.97 MiB | 6.34 MiB/s, done.
Resolving deltas: 100% (7/7), done.
Note: checking out '2e5289aee64085bb6a2ddaea46791ebf6f3b3f93'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

: Package-Builder is setting version
: Package-Builder is building package for Python 3.7
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
INFO:conda_build.variants:Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from config.variant
INFO:conda_build.metadata:Attempting to finalize metadata for ccc
WARNING: symlink_conda() is deprecated.
zip_safe flag not set; analyzing archive contents...
ccc.__pycache__.data.cpython-37: module references __file__
ccc.__pycache__.parameters.cpython-37: module references __file__
ccc.__pycache__.utils.cpython-37: module references __file__
INFO:conda_build.build:Packaging ccc
INFO:conda_build.build:Packaging ccc-1.0.0-py37_0
nm: dynamic_annotations.o: no symbols
Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
WARNING:conda_build.build:Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
INFO:conda_build.variants:Adding in variants from /tmp/tmpjn_ixttb/info/recipe/conda_build_config.yaml
WARNING: symlink_conda() is deprecated.
Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/test_tmp/run_test.py", line 2, in <module>
    import ccc
ModuleNotFoundError: No module named 'ccc'
Tests failed for ccc-1.0.0-py37_0.tar.bz2 - moving package to /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/broken
WARNING:conda_build.build:Tests failed for ccc-1.0.0-py37_0.tar.bz2 - moving package to /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/broken
TESTS FAILED: ccc-1.0.0-py37_0.tar.bz2
Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/utils.py", line 20, in os_call
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/subprocess.py", line 487, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'conda build --python 3.7 --old-build-string --channel pslmodels --override-channels --no-anaconda-upload --output-folder pkgbld_output conda.recipe' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/bin/pbrelease", line 11, in <module>
    sys.exit(main())
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/cli.py", line 103, in main
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/release.py", line 189, in release
  File "/home/hdoupe/anaconda3/envs/pkgbld-env/lib/python3.7/site-packages/pkgbld-0.22.0-py3.7.egg/pkgbld/utils.py", line 25, in os_call
OSError: non-zero return code 1 generated by command:
conda build --python 3.7 --old-build-string --channel pslmodels --override-channels --no-anaconda-upload --output-folder pkgbld_output conda.recipe
>output: b"Adding in variants from internal_defaults\nAdding in variants from config.variant\nAttempting to finalize metadata for ccc\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\nBUILD START: ['ccc-1.0.0-py37_0.tar.bz2']\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\n\n## Package Plan ##\n\n  environment location: /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p\n\n\nThe following NEW packages will be INSTALLED:\n\n    asn1crypto:      0.24.0-py37_1003      conda-forge\n    bokeh:           1.1.0-py37_0          conda-forge\n    bzip2:           1.0.6-h14c3975_1002   conda-forge\n    ca-certificates: 2019.3.9-hecc5488_0   conda-forge\n    certifi:         2019.3.9-py37_0       conda-forge\n    cffi:            1.12.3-py37h8022711_0 conda-forge\n    chardet:         3.0.4-py37_1003       conda-forge\n    cryptography:    2.6.1-py37h72c5cf5_0  conda-forge\n    freetype:        2.10.0-he983fc9_0     conda-forge\n    idna:            2.8-py37_1000         conda-forge\n    jinja2:          2.10.1-py_0           conda-forge\n    jpeg:            9c-h14c3975_1001      conda-forge\n    libblas:         3.8.0-10_openblas     conda-forge\n    libcblas:        3.8.0-10_openblas     conda-forge\n    libffi:          3.2.1-he1b5a44_1006   conda-forge\n    libgcc-ng:       8.2.0-hdf63c60_1                 \n    libgfortran-ng:  7.3.0-hdf63c60_0                 \n    liblapack:       3.8.0-10_openblas     conda-forge\n    libpng:          1.6.37-hed695b0_0     conda-forge\n    libstdcxx-ng:    8.2.0-hdf63c60_1                 \n    libtiff:         4.0.10-h648cc4a_1001  conda-forge\n    llvmlite:        0.28.0-py37hdbcaa40_0 conda-forge\n    markupsafe:      1.1.1-py37h14c3975_0  conda-forge\n    marshmallow:     3.0.0rc5-py_0         conda-forge\n    ncurses:         6.1-hf484d3e_1002     conda-forge\n    numba:           0.43.1-py37hf2d7682_0 conda-forge\n    numpy:           1.16.3-py37he5ce36f_0 conda-forge\n    olefile:         0.46-py_0             conda-forge\n    openblas:        0.3.6-h6e990d7_2      conda-forge\n    openssl:         1.1.1b-h14c3975_1     conda-forge\n    packaging:       19.0-py_0             conda-forge\n    pandas:          0.24.2-py37hf484d3e_0 conda-forge\n    paramtools:      0.5.4-py37_0          pslmodels  \n    pillow:          6.0.0-py37he7afcd5_0  conda-forge\n    pip:             19.1-py37_0           conda-forge\n    pycparser:       2.19-py37_1           conda-forge\n    pyopenssl:       19.0.0-py37_0         conda-forge\n    pyparsing:       2.4.0-py_0            conda-forge\n    pysocks:         1.7.0-py37_0          conda-forge\n    python:          3.7.3-h5b0a415_0      conda-forge\n    python-dateutil: 2.8.0-py_0            conda-forge\n    pytz:            2019.1-py_0           conda-forge\n    pyyaml:          5.1-py37h14c3975_0    conda-forge\n    readline:        7.0-hf8c457e_1001     conda-forge\n    requests:        2.22.0-py37_0         conda-forge\n    scipy:           1.3.0-py37hab63836_0  conda-forge\n    setuptools:      41.0.1-py37_0         conda-forge\n    six:             1.12.0-py37_1000      conda-forge\n    sqlite:          3.28.0-h8b20d00_0     conda-forge\n    taxcalc:         2.2.0-py37_0          pslmodels  \n    tk:              8.6.9-h84994c4_1001   conda-forge\n    tornado:         6.0.2-py37h516909a_0  conda-forge\n    urllib3:         1.24.3-py37_0         conda-forge\n    wheel:           0.33.4-py37_0         conda-forge\n    xlrd:            1.2.0-py_0            conda-forge\n    xz:              5.2.4-h14c3975_1001   conda-forge\n    yaml:            0.1.7-h14c3975_1001   conda-forge\n    zlib:            1.2.11-h14c3975_1004  conda-forge\n\nPreparing transaction: ...working... done\nVerifying transaction: ...working... done\nExecuting transaction: ...working... done\nCopying /home/hdoupe/temporary_pkgbld_working_dir/Cost-of-Capital-Calculator to /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work\nsource tree in: /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work\nexport PREFIX=/home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p\nexport SRC_DIR=/home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work\n~/temporary_pkgbld_working_dir/Cost-of-Capital-Calculator ~/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work\nrunning install\nrunning bdist_egg\nrunning egg_info\ncreating ccc.egg-info\nwriting ccc.egg-info/PKG-INFO\nwriting dependency_links to ccc.egg-info/dependency_links.txt\nwriting requirements to ccc.egg-info/requires.txt\nwriting top-level names to ccc.egg-info/top_level.txt\nwriting manifest file 'ccc.egg-info/SOURCES.txt'\nreading manifest file 'ccc.egg-info/SOURCES.txt'\nreading manifest template 'MANIFEST.in'\nwriting manifest file 'ccc.egg-info/SOURCES.txt'\ninstalling library code to build/bdist.linux-x86_64/egg\nrunning install_lib\nrunning build_py\ncreating build\ncreating build/lib\ncreating build/lib/ccc\ncopying ccc/get_taxcalc_rates.py -> build/lib/ccc\ncopying ccc/styles.py -> build/lib/ccc\ncopying ccc/calcfunctions.py -> build/lib/ccc\ncopying ccc/__init__.py -> build/lib/ccc\ncopying ccc/calculator.py -> build/lib/ccc\ncopying ccc/constants.py -> build/lib/ccc\ncopying ccc/utils.py -> build/lib/ccc\ncopying ccc/data.py -> build/lib/ccc\ncopying ccc/controls_callback_script.py -> build/lib/ccc\ncopying ccc/parameters.py -> build/lib/ccc\ncopying ccc/ccc_asset_data.csv -> build/lib/ccc\ncopying ccc/default_parameters.json -> build/lib/ccc\ncopying ccc/records_variables.json -> build/lib/ccc\ncreating build/bdist.linux-x86_64\ncreating build/bdist.linux-x86_64/egg\ncreating build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/get_taxcalc_rates.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/styles.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/ccc_asset_data.csv -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/calcfunctions.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/records_variables.json -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/__init__.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/calculator.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/constants.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/default_parameters.json -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/utils.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/data.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/controls_callback_script.py -> build/bdist.linux-x86_64/egg/ccc\ncopying build/lib/ccc/parameters.py -> build/bdist.linux-x86_64/egg/ccc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/get_taxcalc_rates.py to get_taxcalc_rates.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/styles.py to styles.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/calcfunctions.py to calcfunctions.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/__init__.py to __init__.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/calculator.py to calculator.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/constants.py to constants.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/utils.py to utils.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/data.py to data.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/controls_callback_script.py to controls_callback_script.cpython-37.pyc\nbyte-compiling build/bdist.linux-x86_64/egg/ccc/parameters.py to parameters.cpython-37.pyc\ncreating build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncopying ccc.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO\ncreating dist\ncreating 'dist/ccc-1.0.0-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it\nremoving 'build/bdist.linux-x86_64/egg' (and everything under it)\nProcessing ccc-1.0.0-py3.7.egg\ncreating $PREFIX/lib/python3.7/site-packages/ccc-1.0.0-py3.7.egg\nExtracting ccc-1.0.0-py3.7.egg to $PREFIX/lib/python3.7/site-packages\nAdding ccc 1.0.0 to easy-install.pth file\n\nInstalled $PREFIX/lib/python3.7/site-packages/ccc-1.0.0-py3.7.egg\nProcessing dependencies for ccc==1.0.0\nSearching for xlrd==1.2.0\nBest match: xlrd 1.2.0\nAdding xlrd 1.2.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for scipy==1.3.0\nBest match: scipy 1.3.0\nAdding scipy 1.3.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for taxcalc==2.2.0\nBest match: taxcalc 2.2.0\nAdding taxcalc 2.2.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for pandas==0.24.2\nBest match: pandas 0.24.2\nAdding pandas 0.24.2 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for numpy==1.16.3\nBest match: numpy 1.16.3\nAdding numpy 1.16.3 to easy-install.pth file\nInstalling f2py script to $PREFIX/bin\nInstalling f2py3 script to $PREFIX/bin\nInstalling f2py3.7 script to $PREFIX/bin\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for numba==0.43.1\nBest match: numba 0.43.1\nAdding numba 0.43.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for bokeh==1.1.0\nBest match: bokeh 1.1.0\nAdding bokeh 1.1.0 to easy-install.pth file\nInstalling bokeh script to $PREFIX/bin\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for pytz==2019.1\nBest match: pytz 2019.1\nAdding pytz 2019.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for python-dateutil==2.8.0\nBest match: python-dateutil 2.8.0\nAdding python-dateutil 2.8.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for llvmlite==0.28.0\nBest match: llvmlite 0.28.0\nAdding llvmlite 0.28.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for PyYAML==5.1\nBest match: PyYAML 5.1\nAdding PyYAML 5.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for Pillow==6.0.0\nBest match: Pillow 6.0.0\nAdding Pillow 6.0.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for tornado==6.0.2\nBest match: tornado 6.0.2\nAdding tornado 6.0.2 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for packaging==19.0\nBest match: packaging 19.0\nAdding packaging 19.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for Jinja2==2.10.1\nBest match: Jinja2 2.10.1\nAdding Jinja2 2.10.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for six==1.12.0\nBest match: six 1.12.0\nAdding six 1.12.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for pyparsing==2.4.0\nBest match: pyparsing 2.4.0\nAdding pyparsing 2.4.0 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nSearching for MarkupSafe==1.1.1\nBest match: MarkupSafe 1.1.1\nAdding MarkupSafe 1.1.1 to easy-install.pth file\n\nUsing $PREFIX/lib/python3.7/site-packages\nFinished processing dependencies for ccc==1.0.0\n~/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work\n\nResource usage statistics from building ccc:\n   Process count: 1\n   CPU time: Sys=0:00:00.0, User=0:00:00.0\n   Memory: 3.3M\n   Disk usage: 98.3K\n   Time elapsed: 0:00:02.1\n\nPackaging ccc\nPackaging ccc-1.0.0-py37_0\ncompiling .pyc files...\nfound egg dir: /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib/python3.7/site-packages/ccc-1.0.0-py3.7.egg\nnumber of files: 29\n   INFO (ccc): plugin library package conda-forge::scipy-1.3.0-py37hab63836_0 in requirements/run but it is not used (i.e. it is overdepending or perhaps statically linked? If that is what you want then add it to `build/ignore_run_exports`)\nFixing permissions\nCompressing to /tmp/tmp57sjfzd5/ccc-1.0.0-py37_0.tar.bz2\nTEST START: pkgbld_output/linux-64/ccc-1.0.0-py37_0.tar.bz2\nAdding in variants from /tmp/tmpjn_ixttb/info/recipe/conda_build_config.yaml\nRenaming work directory,  /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work  to  /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/work_moved_ccc-1.0.0-py37_0_linux-64\nCollecting package metadata: ...working... done\nSolving environment: ...working... done\n\n## Package Plan ##\n\n  environment location: /home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol\n\n\nThe following NEW packages will be INSTALLED:\n\n    asn1crypto:      0.24.0-py36_1003      conda-forge                                                                              \n    bokeh:           1.1.0-py36_0          conda-forge                                                                              \n    ca-certificates: 2019.3.9-hecc5488_0   conda-forge                                                                              \n    ccc:             1.0.0-py37_0          file:///home/hdoupe/temporary_pkgbld_working_dir/Cost-of-Capital-Calculator/pkgbld_output\n    certifi:         2019.3.9-py36_0       conda-forge                                                                              \n    cffi:            1.12.3-py36h8022711_0 conda-forge                                                                              \n    chardet:         3.0.4-py36_1003       conda-forge                                                                              \n    cryptography:    2.6.1-py36h72c5cf5_0  conda-forge                                                                              \n    freetype:        2.10.0-he983fc9_0     conda-forge                                                                              \n    idna:            2.8-py36_1000         conda-forge                                                                              \n    jinja2:          2.10.1-py_0           conda-forge                                                                              \n    jpeg:            9c-h14c3975_1001      conda-forge                                                                              \n    libblas:         3.8.0-10_openblas     conda-forge                                                                              \n    libcblas:        3.8.0-10_openblas     conda-forge                                                                              \n    libffi:          3.2.1-he1b5a44_1006   conda-forge                                                                              \n    libgcc-ng:       8.2.0-hdf63c60_1                                                                                               \n    libgfortran-ng:  7.3.0-hdf63c60_0                                                                                               \n    liblapack:       3.8.0-10_openblas     conda-forge                                                                              \n    libpng:          1.6.37-hed695b0_0     conda-forge                                                                              \n    libstdcxx-ng:    8.2.0-hdf63c60_1                                                                                               \n    libtiff:         4.0.10-h648cc4a_1001  conda-forge                                                                              \n    llvmlite:        0.28.0-py36hdbcaa40_0 conda-forge                                                                              \n    markupsafe:      1.1.1-py36h14c3975_0  conda-forge                                                                              \n    marshmallow:     3.0.0rc5-py_0         conda-forge                                                                              \n    ncurses:         6.1-hf484d3e_1002     conda-forge                                                                              \n    numba:           0.43.1-py36hf2d7682_0 conda-forge                                                                              \n    numpy:           1.16.3-py36he5ce36f_0 conda-forge                                                                              \n    olefile:         0.46-py_0             conda-forge                                                                              \n    openblas:        0.3.6-h6e990d7_2      conda-forge                                                                              \n    openssl:         1.1.1b-h14c3975_1     conda-forge                                                                              \n    packaging:       19.0-py_0             conda-forge                                                                              \n    pandas:          0.24.2-py36hf484d3e_0 conda-forge                                                                              \n    paramtools:      0.5.4-py36_0          pslmodels                                                                                \n    pillow:          6.0.0-py36he7afcd5_0  conda-forge                                                                              \n    pip:             19.1-py36_0           conda-forge                                                                              \n    pycparser:       2.19-py36_1           conda-forge                                                                              \n    pyopenssl:       19.0.0-py36_0         conda-forge                                                                              \n    pyparsing:       2.4.0-py_0            conda-forge                                                                              \n    pysocks:         1.7.0-py36_0          conda-forge                                                                              \n    python:          3.6.7-h381d211_1004   conda-forge                                                                              \n    python-dateutil: 2.8.0-py_0            conda-forge                                                                              \n    pytz:            2019.1-py_0           conda-forge                                                                              \n    pyyaml:          5.1-py36h14c3975_0    conda-forge                                                                              \n    readline:        7.0-hf8c457e_1001     conda-forge                                                                              \n    requests:        2.22.0-py36_0         conda-forge                                                                              \n    scipy:           1.3.0-py36hab63836_0  conda-forge                                                                              \n    setuptools:      41.0.1-py36_0         conda-forge                                                                              \n    six:             1.12.0-py36_1000      conda-forge                                                                              \n    sqlite:          3.28.0-h8b20d00_0     conda-forge                                                                              \n    taxcalc:         2.2.0-py36_0          pslmodels                                                                                \n    tk:              8.6.9-h84994c4_1001   conda-forge                                                                              \n    tornado:         6.0.2-py36h516909a_0  conda-forge                                                                              \n    urllib3:         1.24.3-py36_0         conda-forge                                                                              \n    wheel:           0.33.4-py36_0         conda-forge                                                                              \n    xlrd:            1.2.0-py_0            conda-forge                                                                              \n    xz:              5.2.4-h14c3975_1001   conda-forge                                                                              \n    yaml:            0.1.7-h14c3975_1001   conda-forge                                                                              \n    zlib:            1.2.11-h14c3975_1004  conda-forge                                                                              \n\nPreparing transaction: ...working... done\nVerifying transaction: ...working... done\nExecuting transaction: ...working... done\nexport PREFIX=/home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol\nexport SRC_DIR=/home/hdoupe/anaconda3/envs/pkgbld-env/conda-bld/ccc_1558637908969/test_tmp\nimport: 'ccc'\n"
(pkgbld-env) hdoupe@pop-os:~$ 
hdoupe commented 5 years ago

I was able to build CCC locally after removing the python version specification in the run section of the meta.yaml file https://github.com/PSLmodels/Cost-of-Capital-Calculator/compare/1.0.0...hdoupe:fix-pyver-v100.

(ccc-exper-env) hdoupe@pop-os:~/Documents/Cost-of-Capital-Calculator$ pbrelease Cost-of-Capital-Calculator ccc 1.0.0 --local
: Package-Builder will build model packages for:
:   repository_name = Cost-of-Capital-Calculator
:   package_name = ccc
:   model_version = 1.0.0
:   python_versions = ['3.7']
: Package-Builder will install package on local computer
: Package-Builder is starting at Thu May 23 15:09:50 2019
: Package-Builder is copying local source code
: Package-Builder is setting version
: Package-Builder is building package for Python 3.7
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.11
INFO:conda_build.variants:Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from config.variant
INFO:conda_build.metadata:Attempting to finalize metadata for ccc
WARNING: symlink_conda() is deprecated.
zip_safe flag not set; analyzing archive contents...
ccc.__pycache__.data.cpython-37: module references __file__
ccc.__pycache__.parameters.cpython-37: module references __file__
ccc.__pycache__.utils.cpython-37: module references __file__
INFO:conda_build.build:Packaging ccc
INFO:conda_build.build:Packaging ccc-1.0.0-py37_0
nm: dynamic_annotations.o: no symbols
Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
WARNING:conda_build.build:Importing conda-verify failed.  Please be sure to test your packages.  conda install conda-verify to make this message go away.
INFO:conda_build.variants:Adding in variants from /tmp/tmp0077emul/info/recipe/conda_build_config.yaml
WARNING: symlink_conda() is deprecated.
: Package-Builder is uninstalling any existing package

PackagesNotFoundError: The following packages are missing from the target environment:
  - ccc

: Package-Builder is installing package on local computer
: Package-Builder is cleaning-up
: Package-Builder is finishing at Thu May 23 15:13:18 2019
(ccc-exper-env) hdoupe@pop-os:~/Documents/Cost-of-Capital-Calculator$ cd ..
(ccc-exper-env) hdoupe@pop-os:~/Documents$ python
Python 3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ccc
>>> ccc.__version__
'1.0.0'
>>> ccc.__file__
'/home/hdoupe/anaconda3/envs/ccc-exper-env/lib/python3.7/site-packages/ccc/__init__.py'
>>> 
(ccc-exper-env) hdoupe@pop-os:~/Documents$ cd Cost-of-Capital-Calculator/
hdoupe commented 5 years ago

CCC was pinned to Python 3.6 at run time. That could explain why CCC could be built but the run-time test (import ccc) would fail.

martinholmer commented 5 years ago

@hdoupe said:

I was able to build CCC locally after removing the python version specification in the run section of the meta.yaml file PSLmodels/Cost-of-Capital-Calculator@1.0.0...hdoupe:fix-pyver-v100.

CCC was pinned to Python 3.6 at run time. That could explain why CCC could be built but the run-time test (import ccc) would fail [on my computer running Python 3.7].

Thanks for seeing this through to the end. It is good to clarify what was the source of the problem: an inappropriate pin in the Cost-of-Capital-Calculator/conda.build/meta.yaml file in release 1.0.0.

Let me clarify one last thing. When you executed this command:

pbrelease Cost-of-Capital-Calculator ccc 1.0.0 --local

is it true that you had previously executed this command:

conda config add --channels conda-forge

That's what it looks like to me. The above conda config add --channels conda-forge command makes the enhancement you developed in Package-Builder PR #168 unnecessary, right?

hdoupe commented 5 years ago

@martinholmer asked:

Let me clarify one last thing. When you executed this command:

pbrelease Cost-of-Capital-Calculator ccc 1.0.0 --local

is it true that you had previously executed this command:

conda config add --channels conda-forge

Yes, conda-forge is on my channels list. Let's close this PR.

Thanks for seeing this through to the end.

Sure thing. Thanks for your help!

@jdebacker you should be good to go once you update your meta.yaml file to remove the Python pin.

jdebacker commented 5 years ago

Thanks @hdoupe and @martinholmer. The meta.yml was already updated to remove the pin with @martinholmer 's recent PR to CCC. Now I'll add conda-forge to my list of channels and look at bringing ParamTools back to CCC.