PSLmodels / Package-Builder

Policy Simulation Library tool that builds packages and uploads them to the Anaconda Cloud's PSLmodels channel for public distribution
4 stars 7 forks source link

Add simple script for releasing packages #81

Closed hdoupe closed 6 years ago

hdoupe commented 6 years ago

This PR proposes a simple script that does most of the things that this repo does but in a simpler way.

Advantages:

Disadvantages:

@martinholmer @jdebacker Does this script look like it could be useful for you? Does my analysis of the pros/cons seem sensible?

Also, I successfully released OG-USA 0.5.11 with this script.

martinholmer commented 6 years ago

@hdoupe, a quick review of PR #81 leaves me with two questions:

  1. Is the following comment still needed after you've finished development?
    # need to swap out the 0.5.10 for vers
  2. Given your emphasis on code clarity (which is excellent), wouldn't it be clearer if you renamed the build function as build_and_upload?

I'm not sure what my answer is to your question about whether or not this script would be useful to me. I've had no problems using the pb CLI to this repository, so there's little incentive for me to change. But others coming to the conda package build-and-upload task for the first time may well opt for your script.

martinholmer commented 6 years ago

@hdoupe, one more question about PR #81, the high-level logic for which is here:

if __name__ == '__main__':
    owner, repo, package, vers = sys.argv[1:]
    print('owner', owner)
    print('repo name', repo)
    print('package name', package
    print('package version', vers)
    run(f'git clone https://github.com/{owner}/{repo}/')
    os.chdir(repo)
    run(f'git checkout -b v{vers} {vers}')
    replace_version(vers)
    run(f'conda config --add channels ospc')
    if not os.path.isdir('artifacts'):
        os.mkdir('artifacts')
    for python_version in ('2.7', '3.6'):
        build(python_version, repo, package, vers)

Why this line? run(f'git clone https://github.com/{owner}/{repo}/') Why make a release from a user's fork of the repository? Isn't that dangerous? How do we know the user's fork of the upstream GitHub repository is in sync with the master branch of the main GitHub repository? Why not just clone the main GitHub repository? Maybe there are sensible answers to these questions, but we need to know why you're approaching it this way.

@jdebacker @MattHJensen

hdoupe commented 6 years ago

@martinholmer thanks for reviewing this. I removed the stale comment and updated the function name. I'm glad you think that the code is clear. That was one of my main goals--remove the complexity at the expense of a few bells and whistles.

Why this line? run(f'git clone https://github.com/{owner}/{repo}/') Why make a release from a user's fork of the repository? Isn't that dangerous? How do we know the user's fork of the upstream GitHub repository is in sync with the master branch of the main GitHub repository? Why not just clone the main GitHub repository? Maybe there are sensible answers to these questions, but we need to know why you're approaching it this way.

These are good points. I didn't consider these when I wrote this. However, I built and uploaded the OG-USA script with the following command: python builder.py open-source-economics OG-USA ogusa 0.5.11 which clones the upstream repo. For now, I can add a comment in the script that recommends using the upstream repo. Does that seem sensible?

Also, I used "owner" throughout the script for lack of a better term. Do you have a better term than "owner"? i.e. what should the "open-source-economics" github umbrella repo be called?

hdoupe commented 6 years ago

@martinholmer said:

I'm not sure what my answer is to your question about whether or not this script would be useful to me. I've had no problems using the pb CLI to this repository, so there's little incentive for me to change. But others coming to the conda package build-and-upload task for the first time may well opt for your script.

This makes sense to me. I've had a few headaches using it. Each time it's come down to some kind of user error on my part. When releasing OG-USA this time, I think I did a few things wrong. Also, I screwed up my local anaconda installation which further complicated things. It's just been frustrating not understanding how this repo works when I am having trouble getting it to work. In that light, I am ambivalent about merging this script. However, if it's not merged here, I can still post it in another repo under my own username because I think others may find it useful, too.

martinholmer commented 6 years ago

@hdoupe, after you changes today, the builder.py docstring looks like this:

Release a package with the following commands:
export TOKEN=yourtoken
python builder.py gh-username repo-name package-name package-version

e.g. python builder.py open-source-economics OG-USA ogusa 0.5.11

WARNING: It is highly recommended that the upstream repository is cloned for
the purposes of building and packaging instead of a user's fork of it.
If a user's fork is used, then it may not be up-to-date with the most recent
changes.

Note: `CURRENT_DIST` will have to be changed to your computer's operating
system

This is much better than before, but it raises the question of why user is even an argument. When would anybody be building and uploading to the ospc channel at anaconda.org, a package based on code other than what is in the GitHub repo owned by the open-source-economics user. Especially because that repo is the one that will have the release info and source code download for each release. It seems to me that all these things go together.

Why not drop the owner and hard-wire open-source-economics in its place?

@jdebacker @MattHJensen

martinholmer commented 6 years ago

@hdoupe included this line in the new builder.py script:

DISTS = {'linux-32', 'linux-64', 'win-32', 'win-64', 'osx-64'}

Why are we building a package for linux-32? There are no linux-32 packages for either taxcalc or btax. And the prior version of ogusa (0.5.10) had no linux-32 packages.

Is anybody out there running a 32-bit Linux operating system?

Now that I ask that question, I guess we should ask ourselves: is anybody out there running a 32-bit Windows operating system? If not, why are all the OSPC repos generating win-32 packages?

And also, why does ogusa version 0.5.10 have only Python 3.6 packages?

@jdebacker @MattHJensen

hdoupe commented 6 years ago

@martinholmer asked:

Why not drop the owner and hard-wire open-source-economics in its place?

I don't have any firm preferences about this. I allowed "user" to be specified because I figured that anyone who wants to release a conda package could use this, not just the open-source-economics group.

Why are we building a package for linux-32? There are no linux-32 packages for either taxcalc or btax. And the prior version of ogusa (0.5.10) had no linux-32 packages.

A package was built for linux-32 because I pulled the distribution list from this page: screen shot 2018-06-05 at 9 17 54 am https://anaconda.org/ospc/taxcalc

Now that I look at this more carefully, I can see that there hasn't been a linux-32 release since Tax-Calculator 0.15.2.

And also, why does ogusa version 0.5.10 have only Python 3.6 packages?

Sorry, that's my fault. OG-USA 0.5.10 didn't build correctly. I made some changes and a new version 0.5.11 was released. However, I hard coded 0.5.10 into the OG-USA meta.yaml without understanding that I couldn't override it with the anaconda command line tool when I did the release. So, in the process of trying to get the anaconda command line tool to work, I released OG-USA 0.5.11 but it was tagged as 0.5.10 (because that's what was in the meta.yaml). I was using Python 3.6 when these efforts were made. Thus, that's why 0.5.10 is only available in Python 3.6. I'll clean that up. There shouldn't be an OG-USA 0.5.10 on conda because it didn't build correctly.

(RE specifying the version: what actually happens (both in the current system and in my implementation) is the version in the meta.yaml is replaced and the edited meta.yaml is saved.)

Is anybody out there running a 32-bit Linux operating system?

Now that I ask that question, I guess we should ask ourselves: is anybody out there running a 32-bit Windows operating system? If not, why are all the OSPC repos generating win-32 packages?

I'm not sure what the answer is to these questions. This article appears to have a lot of useful information about this. It seems like most computers are 64 bit these days but there could potentially be old machines that still run 32 bit. However, my knowledge about this is extremely limited.

martinholmer commented 6 years ago

@hdoupe, I think we can safely remove packages for linux-32. Nobody has ever download an OSPC package like that in two years.

But Microsoft still offers a 32-bit version of Windows 10 for some odd reason, so we should keep win-32. I'll removelinux-32 from the master branch; you remove it from your pull request.

hdoupe commented 6 years ago

@martinholmer said:

@hdoupe, I think we can safely remove packages for linux-32. Nobody has ever download an OSPC package like that in two years.

But Microsoft still offers a 32-bit version of Windows 10 for some odd reason, so we should keep win-32. I'll removelinux-32 from the master branch; you remove it from your pull request.

Agreed. The most recent commit fec7328 removes linux-32.

martinholmer commented 6 years ago

@hdoupe said:

Why not drop the owner and hard-wire open-source-economics in its place?

I don't have any firm preferences about this. I allowed "user" to be specified because I figured that anyone who wants to release a conda package could use this [that is, the builder.py script], not just the open-source-economics group.

It's good to make the script as general as possible, but it already requires editing to specify the user's OS, as you say in this line from the docstring:

Note: `CURRENT_DIST` will have to be changed to your computer's operating system

How about making the following changes before we merge this pull request?

  1. Make the lines right below the docstring look something like the following and use these three global variables in the builder.py code.

    GITHUB_ORGANIZATION = 'open-source-economics'
    PYTHON_VERSIONS = ['2.7', '3.6']
    OPERATING_SYSTEMS = ['linux-64', 'win-32', 'win-64', 'osx-64']
  2. Make the current operating system, which is now hard-wired as CURRENT_DIST, be specified automatically by using the Python platform capabilities.

hdoupe commented 6 years ago

@martinholmer The most recent commits implement your suggestions.

I'm not sure what the best way is to get the current OS. The get_current_os function seems to work for Mac and Linux (Ubuntu flavor). Do you have access to a windows machine to test it on? Also do you have any other ideas for how to get the current OS?

martinholmer commented 6 years ago

@hdoupe, thanks for the recent changes . After making those changes, is there any need for the following lines in the docstring?

WARNING: It is highly recommended that the upstream repository is cloned for
the purposes of building and packaging instead of a user's fork of it.
If a user's fork is used, then it may not be up-to-date with the most recent
changes.
hdoupe commented 6 years ago

@martinholmer nice catch. I dropped the warning message.

What do you think so far? Do you think it would be helpful to add a simple cli argument parser with argparse?

martinholmer commented 6 years ago

@hdoupe, I think there's a need to add a space after ** on this new line:

 is_64bit = sys.maxsize > 2 **32
hdoupe commented 6 years ago

Yep, thanks @martinholmer

martinholmer commented 6 years ago

@hdoupe, I don't have access to Windows and I'm not sure anybody on the OSPC staff has access to a Windows machine. But your get_current_os() code looks sensible to me. I'm fine with this part now.

martinholmer commented 6 years ago

@hdoupe asked:

What do you think so far? Do you think it would be helpful to add a simple cli argument parser with argparse?

I'm warming to the simplicity of the builder.py script. I'm glad you pursued this. And after it is merged I'm going to try using it some.

About the argparse enhancements: frankly I don't think the effort is worth it. What would be added?

What I do think needs to be done is to fix this line:

 run(f'conda config --add channels ospc')

If anybody else (outside of OSPC) runs this script they will be trying to upload the packages to the ospc channel. How about at the top:

CONDA_CHANNEL = 'ospc'

And then revise that one line to look something like this:

 run(f'conda config --add channels {CONDA_CHANNEL}')
martinholmer commented 6 years ago

@hdoupe, why two different global variables that are “ospc”? When would they be different?

hdoupe commented 6 years ago

@martinholmer said:

I'm warming to the simplicity of the builder.py script. I'm glad you pursued this. And after it is merged I'm going to try using it some.

Great, I hope you find it to be helpful. I'm sure your use of this will help with finding bugs and polishing up the script.

About the argparse enhancements: frankly I don't think the effort is worth it. What would be added?

Agreed, there aren't enough arguments to justify setting it up.

What I do think needs to be done is to fix this line:

run(f'conda config --add channels ospc')

If anybody else (outside of OSPC) runs this script they will be trying to upload the packages to the ospc channel. How about at the top:

CONDA_CHANNEL = 'ospc'

And then revise that one line to look something like this:

run(f'conda config --add channels {CONDA_CHANNEL}')

Ah, good point. I forgot about that. Here's what I tried to do in commit 3dc55fd:

I think the TOKEN argument in anaconda upload is what determines the channel to which the package will be uploaded. However, this is just a guess. I'm having trouble finding documentation on the anaconda upload command.

hdoupe commented 6 years ago

@martinholmer does the above comment answer your question:

@hdoupe, why two different global variables that are “ospc”? When would they be different?

edit: My take is that DEP_CONDA_CHANNEL and CONDA_USER are different but have the same value by coincidence. DEP_CONDA_CHANNEL is the ospc anaconda channel (or any other non-default conda channel). CONDA_USER is the user name that is set to the upload. The file upload history shows that all uploads are by this 'ospc' user.

martinholmer commented 6 years ago

@hdoupe, OK. It all seems to be complicated. What does “DEP_” stand for?

hdoupe commented 6 years ago

@hdoupe, OK. It all seems to be complicated. What does “DEP_” stand for?

I know...I'm happy to sacrifice generality for simplicity if we are unable to find a simpler solution for this.

DEP_CONDA_CHANNEL stands for dependent conda channel

martinholmer commented 6 years ago

What’s a “dependent conda channel”?

hdoupe commented 6 years ago

What’s a “dependent conda channel”?

It's the channel where the dependent package can be found. For ogusa the only package that cannot be found in the default channel is taxcalc. taxcalc can be found in the ospc channel. Thus, the "dependent conda channel" is ospc.

All of Tax-Calculator's dependencies can be found in the default channel. Thus, the DEP_CONDA_CHANNEL could be set to an empty string or similar.

Does this make sense? Is there a variable name that you think would be more descriptive?

martinholmer commented 6 years ago

@hdoupe said:

It's the channel where the dependent package can be found. For ogusa the only package that cannot be found in the default channel is taxcalc. taxcalc can be found in the ospc channel. Thus, the "dependent conda channel" is ospc.

All of Tax-Calculator's dependencies can be found in the default channel. Thus, the DEP_CONDA_CHANNEL could be set to an empty string or similar.

Does this make sense?

Not really, but that's probably because I don't know much about making conda packages for btax or ogusa.

When making an ogusa package, why does conda need to know where the taxcalc package is?
Where it is has already been specified in the OG-USA environment.yml file, right? And the ogusa package certainly does not include the taxcalc package because the ogusa packages (which are less than one megabyte in size) are much smaller than the taxcalc package (which is about 38 megabytes in size because it includes large data files).

So, as you can see, I'm pretty confused. So confused I can't yet offer a suggestion about an alternative name for the DEP_CONDA_CHANNEL variable.

martinholmer commented 6 years ago

@hdoupe, In the latest version of PR #81, the builder.py script specifies a CONDA_USER variable, but I can't seem to find in the builder.py code where that variable is used. Am I missing something?

hdoupe commented 6 years ago

@martinholmer Sorry for causing this confusion. Let me try this again.

To install Tax-Calculator using conda, you have to specify the channel where conda can find the package. For example, this is what happens when you try to install Tax-Calculator in a fresh environment:

HDoupe-MacBook-Pro:PolicyBrain henrydoupe$ docker run -it continuumio/miniconda3
Unable to find image 'continuumio/miniconda3:latest' locally
latest: Pulling from continuumio/miniconda3
c73ab1c6897b: Already exists 
8bb6d614ca90: Pull complete 
5c852c8f9469: Pull complete 
558b0b4a5dc6: Pull complete 
cc78d8d033c8: Pull complete 
Digest: sha256:177e29f2ffdf1ef1bc4c839f6f55706c2c7f5b6db8dd492964b1487d16466cb2
Status: Downloaded newer image for continuumio/miniconda3:latest
(base) root@b06ae2dc8f46:/# python --version
Python 3.6.4 :: Anaconda, Inc.
(base) root@b06ae2dc8f46:/# conda install taxcalc # this fails
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - taxcalc

Current channels:

  - https://repo.continuum.io/pkgs/main/linux-64
  - https://repo.continuum.io/pkgs/main/noarch
  - https://repo.continuum.io/pkgs/free/linux-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/linux-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/linux-64
  - https://repo.continuum.io/pkgs/pro/noarch

(base) root@b06ae2dc8f46:/# conda install taxcalc -c ospc # this works
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.4

Please update conda by running

    $ conda update -n base conda

## Package Plan ##

  environment location: /opt/conda

  added / updated specs: 
    - taxcalc

...

I don't have a full understanding of how conda packaging works, but from my observations, conda needs to install all of the dependent packages in order to build the target package. The output from installing these packages is present in the logs from a build. So, when OG-USA is built, somewhere in the process, a command equivalent to conda install taxcalc is run. If conda is not configured to look in the OSPC channel, then conda will not find taxcalc. The command conda config --add channels ospc command does something like placing 'ospc' in the list of channels where conda will check for packages:

HDoupe-MacBook-Pro:PolicyBrain henrydoupe$ docker run -it continuumio/miniconda3
(base) root@baa4b44da93f:/# conda install taxcalc # this fails
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - taxcalc

Current channels:

  - https://repo.continuum.io/pkgs/main/linux-64
  - https://repo.continuum.io/pkgs/main/noarch
  - https://repo.continuum.io/pkgs/free/linux-64
  - https://repo.continuum.io/pkgs/free/noarch
  - https://repo.continuum.io/pkgs/r/linux-64
  - https://repo.continuum.io/pkgs/r/noarch
  - https://repo.continuum.io/pkgs/pro/linux-64
  - https://repo.continuum.io/pkgs/pro/noarch

(base) root@baa4b44da93f:/# conda config --add channels ospc
(base) root@baa4b44da93f:/# conda install taxcalc # this works
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.4.10
  latest version: 4.5.4

Please update conda by running

    $ conda update -n base conda

## Package Plan ##

  environment location: /opt/conda

  added / updated specs: 
    - taxcalc

...

Here's a full example which I hope will tie this all together: command summary:

(base) root@b92c49598989:/OG-USA# history
    1  conda install conda-build
    2  git clone https://github.com/open-source-economics/OG-USA/
    3  apt --yes update
    4  apt --yes install libgl1-mesa-glx # install requirements for matplotlib
    5  apt --yes install vim # need to change the version in meta.yaml
    6  pip install pyqt5 # req for matplotlib install
    7  cd OG-USA/
    8  vim conda.recipe/meta.yaml 
    9  git checkout -b v0.5.11 0.5.11
   10  conda build conda.recipe/ # fails because taxcalc is not found
   11  conda config --add channels ospc
   12  conda build conda.recipe/ # passes because taxcalc is found, downloaded, and installed

output from key commands:

(base) root@b92c49598989:/OG-USA# conda build conda.recipe/
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for ogusa
INFO:conda_build.metadata:Attempting to finalize metadata for ogusa
Solving environment: ...working... failed

Leaving build/test directories:
  Work:
 /opt/conda/conda-bld/work 
  Test:
 /opt/conda/conda-bld/test_tmp 
Leaving build/test environments:
  Test:
source activate  /opt/conda/conda-bld/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac 
  Build:
source activate  /opt/conda/conda-bld/_build_env 

Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/conda_build/environ.py", line 717, in get_install_actions
    actions = install_actions(prefix, index, specs, force=True)
  File "/opt/conda/lib/python3.6/site-packages/conda/common/io.py", line 46, in decorated
    return f(*args, **kwds)
  File "/opt/conda/lib/python3.6/site-packages/conda/plan.py", line 541, in install_actions
    txn = solver.solve_for_transaction(prune=prune, ignore_pinned=not pinned)
  File "/opt/conda/lib/python3.6/site-packages/conda/core/solve.py", line 505, in solve_for_transaction
    force_remove, force_reinstall)
  File "/opt/conda/lib/python3.6/site-packages/conda/core/solve.py", line 438, in solve_for_diff
    final_precs = self.solve_final_state(deps_modifier, prune, ignore_pinned, force_remove)
  File "/opt/conda/lib/python3.6/site-packages/conda/core/solve.py", line 318, in solve_final_state
    conflicting_specs = r.get_conflicting_specs(tuple(final_environment_specs))
  File "/opt/conda/lib/python3.6/site-packages/conda/resolve.py", line 705, in get_conflicting_specs
    reduced_index = self.get_reduced_index(specs)
  File "/opt/conda/lib/python3.6/site-packages/conda/resolve.py", line 254, in get_reduced_index
    specs, features = self.verify_specs(specs)
  File "/opt/conda/lib/python3.6/site-packages/conda/resolve.py", line 172, in verify_specs
    raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound: 
  - taxcalc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/conda-build", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 420, in main
    execute(sys.argv[1:])
  File "/opt/conda/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 411, in execute
    verify=args.verify)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/api.py", line 200, in build
    notest=notest, need_source_download=need_source_download, variants=variants)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/build.py", line 2177, in build_tree
    notest=notest,
  File "/opt/conda/lib/python3.6/site-packages/conda_build/build.py", line 1202, in build
    output_metas = expand_outputs([(m, need_source_download, need_reparse_in_env)])
  File "/opt/conda/lib/python3.6/site-packages/conda_build/render.py", line 666, in expand_outputs
    for (output_dict, m) in _m.get_output_metadata_set(permit_unsatisfiable_variants=False):
  File "/opt/conda/lib/python3.6/site-packages/conda_build/metadata.py", line 1969, in get_output_metadata_set
    bypass_env_check=bypass_env_check)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/metadata.py", line 698, in finalize_outputs_pass
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/render.py", line 456, in finalize_metadata
    exclude_pattern)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/render.py", line 333, in add_upstream_pins
    permit_unsatisfiable_variants, exclude_pattern)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/render.py", line 321, in _read_upstream_pin_files
    permit_unsatisfiable_variants=permit_unsatisfiable_variants)
  File "/opt/conda/lib/python3.6/site-packages/conda_build/render.py", line 124, in get_env_dependencies
    channel_urls=tuple(m.config.channel_urls))
  File "/opt/conda/lib/python3.6/site-packages/conda_build/environ.py", line 719, in get_install_actions
    raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'taxcalc'}
(base) root@b92c49598989:/OG-USA# conda config --add channels ospc
(base) root@b92c49598989:/OG-USA# conda build conda.recipe/
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
Attempting to finalize metadata for ogusa
INFO:conda_build.metadata:Attempting to finalize metadata for ogusa
Solving environment: ...working... done
Solving environment: ...working... done
BUILD START: ['ogusa-0.5.11-py36_0.tar.bz2']
Solving environment: ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/conda-bld/ogusa_1528467281104/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho

The following NEW packages will be INSTALLED:

    blas:             1.0-mkl                     
    bokeh:            0.12.16-py36_0              
    ca-certificates:  2018.03.07-0                
    certifi:          2018.4.16-py36_0            
    cycler:           0.10.0-py36h93f1223_0       
    dbus:             1.13.2-h714fa37_1           
    expat:            2.2.5-he0dffb1_0            
    fontconfig:       2.12.6-h49f89f6_0           
    freetype:         2.8-hab7d2ae_1              
    glib:             2.56.1-h000015b_0           
    gst-plugins-base: 1.14.0-hbbd80ab_1           
    gstreamer:        1.14.0-hb453b48_1           
    icu:              58.2-h9c2bf20_1             
    intel-openmp:     2018.0.3-0                  
    jinja2:           2.10-py36ha16c418_0         
    jpeg:             9b-h024ee3a_2               
    kiwisolver:       1.0.1-py36h764f252_0        
    libedit:          3.1.20170329-h6b74fdf_2     
    libffi:           3.2.1-hd88cf55_4            
    libgcc-ng:        7.2.0-hdf63c60_3            
    libgfortran-ng:   7.2.0-hdf63c60_3            
    libopenblas:      0.2.20-h9ac9557_7           
    libpng:           1.6.34-hb9fc6fc_0           
    libstdcxx-ng:     7.2.0-hdf63c60_3            
    libxcb:           1.13-h1bed415_1             
    libxml2:          2.9.8-h26e45fe_1            
    llvmlite:         0.23.1-py36hdbcaa40_0       
    markupsafe:       1.0-py36hd9260cd_1          
    matplotlib:       2.2.2-py36h0e671d2_1        
    mkl:              2018.0.3-1                  
    ncurses:          6.1-hf484d3e_0              
    numba:            0.38.0-py36h637b7d7_0       
    numpy:            1.14.3-py36h28100ab_2       
    numpy-base:       1.14.3-py36hdbf6ddf_2       
    openssl:          1.0.2o-h20670df_0           
    packaging:        17.1-py36_0                 
    pandas:           0.23.0-py36h637b7d7_0       
    pcre:             8.42-h439df22_0             
    pip:              10.0.1-py36_0               
    pyparsing:        2.2.0-py36hee85983_1        
    pyqt:             5.9.2-py36h751905a_0        
    python:           3.6.5-hc3d631a_2            
    python-dateutil:  2.7.3-py36_0                
    pytz:             2018.4-py36_0               
    pyyaml:           3.12-py36hafb9ca4_1         
    qt:               5.9.5-h7e424d6_0            
    readline:         7.0-ha6073c6_4              
    scipy:            1.1.0-py36hfc37229_0        
    setuptools:       39.2.0-py36_0               
    sip:              4.19.8-py36hf484d3e_0       
    six:              1.11.0-py36h372c433_1       
    sqlite:           3.23.1-he433501_0           
    taxcalc:          0.20.1-py36_0           ospc
    tk:               8.6.7-hc745277_3            
    toolz:            0.9.0-py36_0                
    tornado:          5.0.2-py36_0                
    wheel:            0.31.1-py36_0               
    xz:               5.2.4-h14c3975_4            
    yaml:             0.1.7-had09818_2            
    zlib:             1.2.11-ha838bed_2           

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
source tree in: /opt/conda/conda-bld/ogusa_1528467281104/work
/OG-USA /opt/conda/conda-bld/ogusa_1528467281104/work
running install
running bdist_egg
running egg_info
creating ogusa.egg-info
writing ogusa.egg-info/PKG-INFO
writing dependency_links to ogusa.egg-info/dependency_links.txt
writing top-level names to ogusa.egg-info/top_level.txt
writing manifest file 'ogusa.egg-info/SOURCES.txt'
reading manifest file 'ogusa.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'ogusa.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/ogusa
copying ogusa/macro_output.py -> build/lib/ogusa
copying ogusa/elliptical_u_est.py -> build/lib/ogusa
copying ogusa/TPI_graphs.py -> build/lib/ogusa
copying ogusa/TPI.py -> build/lib/ogusa
copying ogusa/wealth.py -> build/lib/ogusa
copying ogusa/get_micro_data.py -> build/lib/ogusa
copying ogusa/fiscal.py -> build/lib/ogusa
copying ogusa/SS.py -> build/lib/ogusa
copying ogusa/SS_graphs.py -> build/lib/ogusa
copying ogusa/_version.py -> build/lib/ogusa
copying ogusa/household.py -> build/lib/ogusa
copying ogusa/labor.py -> build/lib/ogusa
copying ogusa/__init__.py -> build/lib/ogusa
copying ogusa/aggregates.py -> build/lib/ogusa
copying ogusa/txfunc.py -> build/lib/ogusa
copying ogusa/tax.py -> build/lib/ogusa
copying ogusa/demographics.py -> build/lib/ogusa
copying ogusa/income.py -> build/lib/ogusa
copying ogusa/calibrate.py -> build/lib/ogusa
copying ogusa/utils.py -> build/lib/ogusa
copying ogusa/wealthinit.py -> build/lib/ogusa
copying ogusa/firm.py -> build/lib/ogusa
copying ogusa/parameters.py -> build/lib/ogusa
creating build/lib/ogusa/scripts
copying ogusa/scripts/execute.py -> build/lib/ogusa/scripts
copying ogusa/scripts/__init__.py -> build/lib/ogusa/scripts
copying ogusa/scripts/postprocess.py -> build/lib/ogusa/scripts
copying ogusa/../TxFuncEst_baseline.pkl -> build/lib/ogusa/..
copying ogusa/../TxFuncEst_policy.pkl -> build/lib/ogusa/..
copying ogusa/parameters_metadata.json -> build/lib/ogusa
creating build/lib/ogusa/data
creating build/lib/ogusa/data/ability
copying ogusa/data/ability/FR_wage_profile_tables.xlsx -> build/lib/ogusa/data/ability
creating build/lib/ogusa/data/demographic
copying ogusa/data/demographic/pop_data.csv -> build/lib/ogusa/data/demographic
copying ogusa/data/demographic/demographic_data.csv -> build/lib/ogusa/data/demographic
copying ogusa/data/demographic/mort_rates2011.csv -> build/lib/ogusa/data/demographic
copying ogusa/data/demographic/mortality_rates.csv -> build/lib/ogusa/data/demographic
creating build/lib/ogusa/data/labor
copying ogusa/data/labor/cps_hours_by_age_hourspct.txt -> build/lib/ogusa/data/labor
creating build/lib/ogusa/data/wealth
copying ogusa/data/wealth/scf2007to2013_wealth_age_all_percentiles.csv -> build/lib/ogusa/data/wealth
UPDATING build/lib/ogusa/_version.py
set build/lib/ogusa/_version.py to '0.5.11+0.g007e598.dirty'
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/macro_output.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/elliptical_u_est.py -> build/bdist.linux-x86_64/egg/ogusa
creating build/bdist.linux-x86_64/egg/ogusa/data
creating build/bdist.linux-x86_64/egg/ogusa/data/wealth
copying build/lib/ogusa/data/wealth/scf2007to2013_wealth_age_all_percentiles.csv -> build/bdist.linux-x86_64/egg/ogusa/data/wealth
creating build/bdist.linux-x86_64/egg/ogusa/data/demographic
copying build/lib/ogusa/data/demographic/pop_data.csv -> build/bdist.linux-x86_64/egg/ogusa/data/demographic
copying build/lib/ogusa/data/demographic/demographic_data.csv -> build/bdist.linux-x86_64/egg/ogusa/data/demographic
copying build/lib/ogusa/data/demographic/mort_rates2011.csv -> build/bdist.linux-x86_64/egg/ogusa/data/demographic
copying build/lib/ogusa/data/demographic/mortality_rates.csv -> build/bdist.linux-x86_64/egg/ogusa/data/demographic
creating build/bdist.linux-x86_64/egg/ogusa/data/labor
copying build/lib/ogusa/data/labor/cps_hours_by_age_hourspct.txt -> build/bdist.linux-x86_64/egg/ogusa/data/labor
creating build/bdist.linux-x86_64/egg/ogusa/data/ability
copying build/lib/ogusa/data/ability/FR_wage_profile_tables.xlsx -> build/bdist.linux-x86_64/egg/ogusa/data/ability
copying build/lib/ogusa/TPI_graphs.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/TPI.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/wealth.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/get_micro_data.py -> build/bdist.linux-x86_64/egg/ogusa
creating build/bdist.linux-x86_64/egg/ogusa/scripts
copying build/lib/ogusa/scripts/execute.py -> build/bdist.linux-x86_64/egg/ogusa/scripts
copying build/lib/ogusa/scripts/__init__.py -> build/bdist.linux-x86_64/egg/ogusa/scripts
copying build/lib/ogusa/scripts/postprocess.py -> build/bdist.linux-x86_64/egg/ogusa/scripts
copying build/lib/ogusa/fiscal.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/SS.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/parameters_metadata.json -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/SS_graphs.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/_version.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/household.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/labor.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/__init__.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/aggregates.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/txfunc.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/tax.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/demographics.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/income.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/calibrate.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/utils.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/wealthinit.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/firm.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/ogusa/parameters.py -> build/bdist.linux-x86_64/egg/ogusa
copying build/lib/TxFuncEst_baseline.pkl -> build/bdist.linux-x86_64/egg
copying build/lib/TxFuncEst_policy.pkl -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/ogusa/macro_output.py to macro_output.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/elliptical_u_est.py to elliptical_u_est.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/TPI_graphs.py to TPI_graphs.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/TPI.py to TPI.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/wealth.py to wealth.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/get_micro_data.py to get_micro_data.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/scripts/execute.py to execute.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/scripts/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/scripts/postprocess.py to postprocess.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/fiscal.py to fiscal.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/SS.py to SS.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/SS_graphs.py to SS_graphs.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/_version.py to _version.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/household.py to household.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/labor.py to labor.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/aggregates.py to aggregates.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/txfunc.py to txfunc.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/tax.py to tax.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/demographics.py to demographics.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/income.py to income.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/calibrate.py to calibrate.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/utils.py to utils.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/wealthinit.py to wealthinit.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/firm.py to firm.cpython-36.pyc
byte-compiling build/bdist.linux-x86_64/egg/ogusa/parameters.py to parameters.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying ogusa.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ogusa.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ogusa.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ogusa.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
ogusa.__pycache__.demographics.cpython-36: module references __file__
ogusa.__pycache__.labor.cpython-36: module references __file__
ogusa.__pycache__.parameters.cpython-36: module references __file__
ogusa.__pycache__.txfunc.cpython-36: module references __file__
ogusa.__pycache__.wealth.cpython-36: module references __file__
creating dist
creating 'dist/ogusa-0.5.11+0.g007e598.dirty-py3.6.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing ogusa-0.5.11+0.g007e598.dirty-py3.6.egg
creating /opt/conda/conda-bld/ogusa_1528467281104/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/ogusa-0.5.11+0.g007e598.dirty-py3.6.egg
Extracting ogusa-0.5.11+0.g007e598.dirty-py3.6.egg to /opt/conda/conda-bld/ogusa_1528467281104/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages
Adding ogusa 0.5.11+0.g007e598.dirty to easy-install.pth file

Installed /opt/conda/conda-bld/ogusa_1528467281104/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/ogusa-0.5.11+0.g007e598.dirty-py3.6.egg
Processing dependencies for ogusa==0.5.11+0.g007e598.dirty
Finished processing dependencies for ogusa==0.5.11+0.g007e598.dirty
/opt/conda/conda-bld/ogusa_1528467281104/work

Resource usage statistics from building ogusa:
   Process count: 1
   CPU time: unavailable
   Memory: 2.7M
   Disk usage: 12B
   Time elapsed: 0:00:02.0

Packaging ogusa
INFO:conda_build.build:Packaging ogusa
Packaging ogusa-0.5.11-py36_0
INFO:conda_build.build:Packaging ogusa-0.5.11-py36_0
compiling .pyc files...
found egg dir: /opt/conda/conda-bld/ogusa_1528467281104/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho/lib/python3.6/site-packages/ogusa-0.5.11+0.g007e598.dirty-py3.6.egg
number of files: 67
Fixing permissions
updating: ogusa-0.5.11-py36_0.tar.bz2
TEST START: /opt/conda/conda-bld/linux-64/ogusa-0.5.11-py36_0.tar.bz2
Updating index at /opt/conda/conda-bld/linux-64 to make package installable with dependencies
INFO:conda_build.build:Updating index at /opt/conda/conda-bld/linux-64 to make package installable with dependencies
Updating index at /opt/conda/conda-bld/noarch to make package installable with dependencies
INFO:conda_build.build:Updating index at /opt/conda/conda-bld/noarch to make package installable with dependencies
Adding in variants from /tmp/tmpaftbx5o4/info/recipe/conda_build_config.yaml
INFO:conda_build.variants:Adding in variants from /tmp/tmpaftbx5o4/info/recipe/conda_build_config.yaml
Renaming work directory,  /opt/conda/conda-bld/ogusa_1528467281104/work  to  /opt/conda/conda-bld/ogusa_1528467281104/work_moved_ogusa-0.5.11-py36_0_linux-64
Solving environment: ...working... done

## Package Plan ##

  environment location: /opt/conda/conda-bld/ogusa_1528467281104/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac

The following NEW packages will be INSTALLED:

    atomicwrites:     1.1.5-py36_0                 
    attrs:            18.1.0-py36_0                
    bokeh:            0.12.16-py36_0               
    ca-certificates:  2018.03.07-0                 
    certifi:          2018.4.16-py36_0             
    click:            6.7-py36h5253387_0           
    cloudpickle:      0.5.3-py36_0                 
    cycler:           0.10.0-py36h93f1223_0        
    cytoolz:          0.9.0.1-py36h14c3975_0       
    dask:             0.17.5-py36_0                
    dask-core:        0.17.5-py36_0                
    dbus:             1.13.2-h714fa37_1            
    distributed:      1.21.8-py36_0                
    expat:            2.2.5-he0dffb1_0             
    fontconfig:       2.12.6-h49f89f6_0            
    freetype:         2.8-hab7d2ae_1               
    glib:             2.56.1-h000015b_0            
    gst-plugins-base: 1.14.0-hbbd80ab_1            
    gstreamer:        1.14.0-hb453b48_1            
    heapdict:         1.0.0-py36_2                 
    icu:              58.2-h9c2bf20_1              
    intel-openmp:     2018.0.3-0                   
    jinja2:           2.10-py36ha16c418_0          
    jpeg:             9b-h024ee3a_2                
    kiwisolver:       1.0.1-py36h764f252_0         
    libedit:          3.1.20170329-h6b74fdf_2      
    libffi:           3.2.1-hd88cf55_4             
    libgcc-ng:        7.2.0-hdf63c60_3             
    libgfortran-ng:   7.2.0-hdf63c60_3             
    libopenblas:      0.2.20-h9ac9557_7            
    libpng:           1.6.34-hb9fc6fc_0            
    libstdcxx-ng:     7.2.0-hdf63c60_3             
    libxcb:           1.13-h1bed415_1              
    libxml2:          2.9.8-h26e45fe_1             
    llvmlite:         0.23.1-py36hdbcaa40_0        
    locket:           0.2.0-py36h787c0ad_1         
    markupsafe:       1.0-py36hd9260cd_1           
    matplotlib:       2.2.2-py36h0e671d2_1         
    mkl:              2018.0.3-1                   
    more-itertools:   4.2.0-py36_0                 
    msgpack-python:   0.5.6-py36h6bb024c_0         
    ncurses:          6.1-hf484d3e_0               
    numba:            0.38.0-py36h637b7d7_0        
    numpy:            1.14.3-py36h28100ab_2        
    numpy-base:       1.14.3-py36h0ea5e3f_1        
    ogusa:            0.5.11-py36_0           local
    openssl:          1.0.2o-h20670df_0            
    packaging:        17.1-py36_0                  
    pandas:           0.23.0-py36h637b7d7_0        
    partd:            0.3.8-py36h36fd896_0         
    pcre:             8.42-h439df22_0              
    pip:              10.0.1-py36_0                
    pluggy:           0.6.0-py36hb689045_0         
    psutil:           5.4.5-py36h14c3975_0         
    py:               1.5.3-py36_0                 
    pyparsing:        2.2.0-py36hee85983_1         
    pyqt:             5.9.2-py36h751905a_0         
    pytest:           3.6.0-py36_1                 
    python:           3.6.5-hc3d631a_2             
    python-dateutil:  2.7.3-py36_0                 
    pytz:             2018.4-py36_0                
    pyyaml:           3.12-py36hafb9ca4_1          
    qt:               5.9.5-h7e424d6_0             
    readline:         7.0-ha6073c6_4               
    scipy:            1.1.0-py36hfc37229_0         
    setuptools:       39.2.0-py36_0                
    sip:              4.19.8-py36hf484d3e_0        
    six:              1.11.0-py36h372c433_1        
    sortedcontainers: 2.0.3-py36_0                 
    sqlite:           3.23.1-he433501_0            
    taxcalc:          0.20.1-py36_0           ospc 
    tblib:            1.3.2-py36h34cf8b6_0         
    tk:               8.6.7-hc745277_3             
    toolz:            0.9.0-py36_0                 
    tornado:          5.0.2-py36_0                 
    wheel:            0.31.1-py36_0                
    xz:               5.2.4-h14c3975_4             
    yaml:             0.1.7-had09818_2             
    zict:             0.1.3-py36h3a3bf81_0         
    zlib:             1.2.11-ha838bed_2            

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
import: 'ogusa'
import: 'ogusa.SS'
import: 'ogusa.TPI'
import: 'ogusa.calibrate'
import: 'ogusa.demographics'
import: 'ogusa.elliptical_u_est'
import: 'ogusa.firm'
import: 'ogusa.fiscal'
import: 'ogusa.get_micro_data'
import: 'ogusa.household'
import: 'ogusa.income'
import: 'ogusa.labor'
import: 'ogusa.macro_output'
import: 'ogusa.parameters'
import: 'ogusa.tax'
import: 'ogusa.txfunc'
import: 'ogusa.utils'
import: 'ogusa.wealth'
import: 'ogusa.wealthinit'

Resource usage statistics from testing ogusa:
   Process count: 2
   CPU time: Sys=0:00:00.3, User=0:00:01.6
   Memory: 150.8M
   Disk usage: 12B
   Time elapsed: 0:00:04.1

TEST END: /opt/conda/conda-bld/linux-64/ogusa-0.5.11-py36_0.tar.bz2
Renaming work directory,  /opt/conda/conda-bld/ogusa_1528467281104/work  to  /opt/conda/conda-bld/ogusa_1528467281104/work_moved_ogusa-0.5.11-py36_0_linux-64_main_build_loop
# Automatic uploading is disabled
# If you want to upload package(s) to anaconda.org later, type:

anaconda upload /opt/conda/conda-bld/linux-64/ogusa-0.5.11-py36_0.tar.bz2

# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

anaconda_upload is not set.  Not uploading wheels: []
####################################################################################
Resource usage summary:

Total time: 0:05:06.8
CPU usage: sys=0:00:00.3, user=0:00:01.6
Maximum memory usage observed: 150.8M
Total disk usage observed (not including envs): 24B

####################################################################################
Source and build intermediates have been left in /opt/conda/conda-bld.
There are currently 1 accumulated.
To remove them, you can run the ```conda build purge``` command

In summary: DEP_CONDA_CHANNEL does one thing --> tells conda where to look for packages that are not in the default conda channel so that the target package can be built and packaged.

Note: I ran all of these examples in a miniconda docker container. This ensured that I was using a clean environment.

@martinholmer does this make any more sense?

hdoupe commented 6 years ago

@martinholmer said:

@hdoupe, In the latest version of PR #81, the builder.py script specifies a CONDA_USER variable, but I can't seem to find in the builder.py code where that variable is used. Am I missing something?

Nope, nice catch. I just fixed this.

martinholmer commented 6 years ago

@hdoupe, Thanks for the conda build tutorial. I think I understand how it works now, but it still seems a little convoluted to me.

But there is one more thing. In the builder.py docstring it says this:

- DEP_CONDA_CHANNEL: if package depends on another package in a non-standard
                     channel (for multiple channels use a space delimited
                     string like 'ospc conda-forge')

and the code contains this statement:

# add depenedent channels if specified
    if DEP_CONDA_CHANNEL:
        run(f'conda config --add channels {DEP_CONDA_CHANNEL}')

But when I look at the conda config documentation it says this:

    --prepend KEY VALUE, --add KEY VALUE
              Add one configuration value to the beginning of a list key.
                  ^^^

and the examples of using --add channels all show adding one channel at a time.

So, perhaps DEP_CONDA_CHANNELS should be a list of strings, and if that list exists, we should loop through the list adding one channel at a time. Have you seen any examples of multiple channel strings working with conda config like you show in the docstring?

hdoupe commented 6 years ago

Ah, you are correct. You can only specify one channel (which is a little odd given that 'channels' is plural). Commit 42a2309 fixes this.

Note: It is difficult to test this script since it necessarily uploads to conda. Before it is merged, it should be used to upload a package to conda.

martinholmer commented 6 years ago

@hdoupe, Thanks for all the work developing the builder.py script. You are right in that this is much simpler than the pb CLI tool, and therefore, easier to understand. And following your development work has taught me a lot about conda package building and uploading. So, it's great that you've taken the time to do this.

I plan to use the builder.py script over the next few months whenever I make conda taxcalc packages for each new release, and I'm hoping others will use it as well. If this usage indicates a need for further improvement, we can make those improvements. And after some successful use, I think we could, at a minimum, change the README.md documentation so that it shows, exclusively, how to use the builder.py script. Eventually, we may even consider removing the old, more complex code that supports the pb CLI tool.

Given this plan, it is better to have the builder.py script on the master branch. Hence, I'm merging this pull request now.

@jdebacker @rickecon @MattHJensen

hdoupe commented 6 years ago

@martinholmer Sure, I hope that we find this script to be reliable and useful. I learned a lot about conda packaging in building this and explaining how it works. I'm glad this was educational for you, too.

I plan to use the builder.py script over the next few months whenever I make conda taxcalc packages for each new release, and I'm hoping others will use it as well. If this usage indicates a need for further improvement, we can make those improvements. And after some successful use, I think we could, at a minimum, change the README.md documentation so that it shows, exclusively, how to use the builder.py script. Eventually, we may even consider removing the old, more complex code that supports the pb CLI tool.

This sounds great. This script still has some rough spots. I hope we can smooth them over through your use of the script over the next few months.