search

E3SM-Project / zstash

Long term HPSS archiving tool for E3SM
BSD 3-Clause "New" or "Revised" License
8 stars 11 forks source link

Conda package of zstash 0.4.2 incorrectly includes python_abi #88

Closed xylar closed 4 years ago

xylar commented 4 years ago 
zstash 0.4.2 py_0
-----------------
file name   : zstash-0.4.2-py_0.tar.bz2
name        : zstash
version     : 0.4.2
build       : py_0
build number: 0
size        : 20 KB
subdir      : noarch
url         : https://conda.anaconda.org/e3sm/noarch/zstash-0.4.2-py_0.tar.bz2
md5         : b284b7db42dd55f4eb60cef99d84f4f7
timestamp   : 2020-08-17 21:05:36 UTC
dependencies: 
  - python >=3.6
  - python_abi 3.8.* *_cp38

As a result, zstash cannot be installed for any version of python other than 3.8. E3SM-Unified supports only python 3.6 and 3.7 at the moment.

@chengzhuzhang has also had this problem building packages of e3sm_diags. I do not know what is incorrectly configured in your conda build that means python_abi is included in the package dependencies. This does not happen for me. I will see if I can reproduce this problem building on a Mac in case that is the issue.

xylar commented 4 years ago

This issue https://github.com/conda/conda-build/issues/3956 suggests that the problem might be that you are using conda-build form defaults whereas you really need the version from conda-forge. I would suggest setting channels to strict and doing a conda update on conda and conda-build to make sure you have the right version:

conda config --add channels conda-forge
conda config --set channel_priority strict
conda update -n base conda conda-build

@chengzhuzhang, can you try this as well and see if it fixes the build problem with e3sm_diags?

Once you have built the package locally, use:

conda search --info --use-local zstash

to make sure python_abi is not being included.

xylar commented 4 years ago

I built zstash on a Mac and it still doesn't have python_abi for me, so I don't think it's a platform issue. I think it's an issue of conda-build coming from the wrong channel as stated above. Please always use strict channel priority and make sure conda-forge is the highest priority channel.

xylar commented 4 years ago

I am able to build zstash without python_abi on Linux as well. Here is my base environment:

$ conda list
# packages in environment at /home/xylar/miniconda3:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
anaconda-client           1.7.2                      py_0    conda-forge
astroid                   2.4.2            py37hc8dfbb8_0    conda-forge
attrs                     20.1.0             pyh9f0ad1d_0    conda-forge
babel                     2.8.0                      py_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.1                      py_0    conda-forge
bash-completion           2.11                          0    conda-forge
beautifulsoup4            4.9.1                      py_1    conda-forge
bleach                    3.1.5              pyh9f0ad1d_0    conda-forge
blinker                   1.4                        py_1    conda-forge
boolean.py                3.7                        py_0    conda-forge
brotlipy                  0.7.0           py37h8f50634_1000    conda-forge
bzip2                     1.0.8                h516909a_2    conda-forge
c-ares                    1.16.1               h516909a_0    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2020.6.20        py37hc8dfbb8_0    conda-forge
cffi                      1.14.1           py37h2b28604_0    conda-forge
chardet                   3.0.4           py37hc8dfbb8_1006    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.5.0                      py_0    conda-forge
clyent                    1.2.2                      py_1    conda-forge
conda                     4.8.4            py37hc8dfbb8_2    conda-forge
conda-bash-completion     1.5                           0    conda-forge
conda-build               3.19.2           py37hc8dfbb8_4    conda-forge
conda-env                 2.6.0                         1    conda-forge
conda-forge-build-setup   4.4.23                        0    conda-forge
conda-forge-ci-setup      2.5.3                    py37_0    conda-forge
conda-forge-pinning       2020.08.21.04.46.41               0    conda-forge
conda-package-handling    1.7.0            py37h8f50634_4    conda-forge
conda-smithy              3.7.10                     py_0    conda-forge
conda-verify              3.1.1                 py37_1000    conda-forge
cryptography              3.0              py37hb09aad4_0    conda-forge
curl                      7.71.1               he644dc0_5    conda-forge
dbus                      1.13.6               he372182_0    conda-forge
decorator                 4.4.2                      py_0    conda-forge
defusedxml                0.6.0                      py_0    conda-forge
deprecated                1.2.10             pyh9f0ad1d_0    conda-forge
docutils                  0.16             py37hc8dfbb8_1    conda-forge
entrypoints               0.3             py37hc8dfbb8_1001    conda-forge
expat                     2.2.9                he1b5a44_2    conda-forge
filelock                  3.0.12             pyh9f0ad1d_0    conda-forge
fontconfig                2.13.1            h86ecdb6_1001    conda-forge
freetype                  2.10.2               he06d7ca_0    conda-forge
future                    0.18.2           py37hc8dfbb8_1    conda-forge
geos                      3.8.0                he1b5a44_0    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
git                       2.28.0          pl526h5e3e691_1    conda-forge
gitdb                     4.0.5                      py_0    conda-forge
gitpython                 3.1.7                      py_0    conda-forge
glib                      2.65.0               h6f030ca_0    conda-forge
glob2                     0.7                        py_0    conda-forge
gst-plugins-base          1.14.5               h0935bb2_2    conda-forge
gstreamer                 1.14.5               h36ae1b5_2    conda-forge
icu                       64.2                 he1b5a44_1    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        1.7.0            py37hc8dfbb8_0    conda-forge
importlib_metadata        1.7.0                         0    conda-forge
ipykernel                 5.3.4            py37h43977f1_0    conda-forge
ipython                   7.17.0           py37hc6149b9_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
isodate                   0.6.0                      py_1    conda-forge
isort                     4.3.21           py37hc8dfbb8_1    conda-forge
jedi                      0.17.2           py37hc8dfbb8_0    conda-forge
jeepney                   0.4.3                      py_0    conda-forge
jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
jpeg                      9d                   h516909a_0    conda-forge
jsonschema                3.2.0            py37hc8dfbb8_1    conda-forge
jupyter_client            6.1.6                      py_0    conda-forge
jupyter_core              4.6.3            py37hc8dfbb8_1    conda-forge
keyring                   21.3.0           py37hc8dfbb8_0    conda-forge
krb5                      1.17.1               hfafb76e_2    conda-forge
lazy-object-proxy         1.4.3            py37h8f50634_2    conda-forge
ld_impl_linux-64          2.34                 hc38a660_9    conda-forge
libarchive                3.3.3             hddc7a2b_1008    conda-forge
libclang                  9.0.1           default_hde54327_0    conda-forge
libcurl                   7.71.1               hcdd3856_5    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_0    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 9.3.0               h24d8f2e_15    conda-forge
libgomp                   9.3.0               h24d8f2e_15    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblief                   0.10.1               he1b5a44_0    conda-forge
libllvm9                  9.0.1                he513fc3_1    conda-forge
libnghttp2                1.41.0               hab1572f_1    conda-forge
libpng                    1.6.37               hed695b0_2    conda-forge
libsodium                 1.0.18               h516909a_0    conda-forge
libssh2                   1.9.0                hab1572f_5    conda-forge
libstdcxx-ng              9.3.0               hdf63c60_15    conda-forge
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxkbcommon              0.10.0               he1b5a44_0    conda-forge
libxml2                   2.9.10               hee79883_0    conda-forge
license-expression        1.2                        py_0    conda-forge
lz4-c                     1.9.2                he1b5a44_3    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
markupsafe                1.1.1            py37h8f50634_1    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4           py37h8f50634_1001    conda-forge
msrest                    0.6.18             pyh9f0ad1d_0    conda-forge
nbconvert                 5.6.1            py37hc8dfbb8_1    conda-forge
nbformat                  5.0.7                      py_0    conda-forge
ncurses                   6.2                  he1b5a44_1    conda-forge
nspr                      4.27                 he1b5a44_1    conda-forge
nss                       3.55                 he751ad9_0    conda-forge
numpydoc                  1.1.0              pyh9f0ad1d_0    conda-forge
oauthlib                  3.0.1                      py_0    conda-forge
openssl                   1.1.1g               h516909a_1    conda-forge
packaging                 20.4               pyh9f0ad1d_0    conda-forge
pandoc                    2.10.1               h516909a_0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
parso                     0.7.0              pyh9f0ad1d_0    conda-forge
patchelf                  0.11                 he1b5a44_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
perl                      5.26.2            h516909a_1006    conda-forge
pexpect                   4.8.0            py37hc8dfbb8_1    conda-forge
pickleshare               0.7.5           py37hc8dfbb8_1001    conda-forge
pip                       20.2.2                     py_0    conda-forge
pkginfo                   1.5.0.1                    py_0    conda-forge
prompt-toolkit            3.0.6                      py_0    conda-forge
psutil                    5.7.2            py37h8f50634_0    conda-forge
pthread-stubs             0.4               h14c3975_1001    conda-forge
ptyprocess                0.6.0                   py_1001    conda-forge
py-lief                   0.10.1           py37h3340039_0    conda-forge
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycosat                   0.6.3           py37h8f50634_1004    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pycrypto                  2.6.1           py37h8f50634_1004    conda-forge
pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
pygithub                  1.53                       py_0    conda-forge
pygments                  2.6.1                      py_0    conda-forge
pyjwt                     1.7.1                      py_0    conda-forge
pylint                    2.5.3            py37hc8dfbb8_0    conda-forge
pyopenssl                 19.1.0                     py_1    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyqt                      5.12.3           py37h8685d9f_3    conda-forge
pyqt5-sip                 4.19.18                  pypi_0    pypi
pyqtchart                 5.12                     pypi_0    pypi
pyqtwebengine             5.12.1                   pypi_0    pypi
pyrsistent                0.16.0           py37h8f50634_0    conda-forge
pysocks                   1.7.1            py37hc8dfbb8_1    conda-forge
python                    3.7.6                h357f687_0    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python-libarchive-c       2.9                      py37_0    conda-forge
python_abi                3.7                     1_cp37m    conda-forge
pytz                      2020.1             pyh9f0ad1d_0    conda-forge
pyyaml                    5.3.1            py37h8f50634_0    conda-forge
pyzmq                     19.0.2           py37hac76be4_0    conda-forge
qt                        5.12.5               hd8c4c69_1    conda-forge
qtawesome                 0.7.2              pyh9f0ad1d_0    conda-forge
qtconsole                 4.7.6              pyh9f0ad1d_0    conda-forge
qtpy                      1.9.0                      py_0    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
requests                  2.24.0             pyh9f0ad1d_0    conda-forge
requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
ripgrep                   12.1.1               h516909a_0    conda-forge
rope                      0.17.0             pyh9f0ad1d_0    conda-forge
ruamel.yaml               0.16.10          py37h8f50634_0    conda-forge
ruamel.yaml.clib          0.2.0            py37h8f50634_1    conda-forge
ruamel_yaml               0.15.80         py37h8f50634_1001    conda-forge
scrypt                    0.8.15           py37hb09aad4_0    conda-forge
secretstorage             3.1.2            py37hc8dfbb8_1    conda-forge
setuptools                49.6.0           py37hc8dfbb8_0    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
smmap                     3.0.4              pyh9f0ad1d_0    conda-forge
snowballstemmer           2.0.0                      py_0    conda-forge
soupsieve                 2.0.1                      py_1    conda-forge
sphinx                    3.2.1                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
spyder                    3.3.6                    py37_1    conda-forge
spyder-kernels            0.5.2                    py37_0    conda-forge
sqlite                    3.33.0               h4cf870e_0    conda-forge
testpath                  0.4.4                      py_0    conda-forge
tk                        8.6.10               hed695b0_0    conda-forge
toml                      0.10.1             pyh9f0ad1d_0    conda-forge
toolz                     0.10.0                     py_0    conda-forge
tornado                   6.0.4            py37h8f50634_1    conda-forge
tqdm                      4.48.2             pyh9f0ad1d_0    conda-forge
traitlets                 4.3.3            py37hc8dfbb8_1    conda-forge
typed-ast                 1.4.1            py37h516909a_0    conda-forge
urllib3                   1.25.10                    py_0    conda-forge
vsts-python-api           0.1.22                     py_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
wrapt                     1.11.2           py37h8f50634_0    conda-forge
wurlitzer                 2.0.0            py37hc8dfbb8_1    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zeromq                    4.3.2                he1b5a44_3    conda-forge
zipp                      3.1.0                      py_0    conda-forge
zlib                      1.2.11            h516909a_1007    conda-forge
zstd                      1.4.5                h6597ccf_2    conda-forge

Note that almost every single package is from conda-forge. @forsyth2 and @chengzhuzhang, could you let me know what your base environments look like?

chengzhuzhang commented 4 years ago

Hi @xylar , thanks for helping out. I followed the instructions you had, and was able to build e3sm_diags without python_abi on my Mac!. During that, I had to do conda remove anaconda to fix:

The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  - defaults/osx-64::anaconda==2020.02=py37_0```

Glad this issue is being fixed and I can build e3sm_diags on my machine now.

xylar commented 4 years ago

Great!!! I do think you and @forsyth2 will see some problems go away if you switch to using conda config --set channel_priority strict consistently. You should also probably add this to your documentation wherever appropriate -- your users should be doing the same.

forsyth2 commented 4 years ago

@xylar I sent you some questions on the conda build. The conda update step fails for me.

xylar commented 4 years ago

@forsyth2, it looks like you are using your conda base environment to install lots of packages (because you are using anaconda3). This is not a good practice if you are building your own packages. The base environment should only be for some basics like anaconda-client, conda and conda-build. You should have other environments that you use for running your day-to-day tasks.

Part of the reason for this is that it is not possible to delete the base environment and start over if something goes wrong, as it has for you. Mixing and matching channels (like defaults and conda-forge) also does not work well, which is part of the reason for having strict channel order and preferring conda-forge over defaults.

My suggestion is to not use anaconda3 as you currently are. That might be fine for someone who is not using conda-forge but it doesn't play nice with conda-forge and particularly with conda-forge build. Instead, I would suggest that you install miniconda3 and start over making environments for your daily needs that are not the base environment. Just put conda-build, conda, anaconda-client and a few things like that in your base. Maybe git if you need that from conda-forge.

The same suggestion applies on cori and the other supercomputers. Use your own install of miniconda3 rather than the system anaconda.

@chengzhuzhang, these suggestions may apply to you as well, depending on how you have your own machine set up.

forsyth2 commented 4 years ago

@xylar Unfortunately, I can't seem to get miniconda working. I downloaded the Python 3.8 for Mac version from https://docs.conda.io/en/latest/miniconda.html, but which conda still gives /usr/local/anaconda3/bin/conda. conda env list still shows the environments I've made. I don't think I've often ran conda install in the base environment.

Am I missing a step somewhere? Should I expect to have nothing in conda env list. Is the miniconda command different?

xylar commented 4 years ago

Yes, as part of installing miniconda3, you should have had the option to make it alter your .bashrc or the equivalent to load the base miniconda3 environment by default. If you said no to that, you can still make it happen afterward:

source ~/miniconda3/etc/profile.d/conda.sh
conda init

I think that will work. If not, you may need to explicitly remove whatever steps are in your .bashrc (or equivelent) for anaconda3 init.

If none of that works, you may need to manually add:

source ~/miniconda3/etc/profile.d/conda.sh
conda activate base

to your .bashrc.

xylar commented 4 years ago

I downloaded the Python 3.8 for Mac version from https://docs.conda.io/en/latest/miniconda.html

Did you download the pkg or the bash installer? I would go with the bash installer.

Should I expect to have nothing in conda env list.

I don't think you fully installed Miniconda if nothing is showing up but the important bit is that you definitely aren't loading the conda command from Miniconda yet.

Is the miniconda command different?

No, it's the same command but should be in a different location.

forsyth2 commented 4 years ago

@xylar thanks for the suggestions. I downloaded the bash installer and ran the following commands, with selected output:

$ ./Miniconda3-latest-MacOSX-x86_64.sh 
$  which conda
/opt/miniconda3/bin/conda
$ conda env list
/opt/miniconda3/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
# conda environments:
#
base                  *  /opt/miniconda3
                         /usr/local/anaconda3
$ conda install conda-build conda anaconda-client
$ conda env list
# conda environments:
#
base                  *  /opt/miniconda3
                         /usr/local/anaconda3
$ conda list
# packages in environment at /opt/miniconda3:
#
# Name                    Version                   Build  Channel
anaconda-client           1.7.2                      py_0    conda-forge
attrs                     20.1.0             pyh9f0ad1d_0    conda-forge
beautifulsoup4            4.9.1                      py_1    conda-forge
brotlipy                  0.7.0           py38h64e0658_1000    conda-forge
bzip2                     1.0.8                haf1e3a3_3    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2020.6.20        py38h32f6830_0    conda-forge
cffi                      1.14.1           py38hc4dd44e_0    conda-forge
chardet                   3.0.4           py38h32f6830_1006    conda-forge
clyent                    1.2.2                      py_1    conda-forge
conda                     4.8.4            py38h32f6830_2    conda-forge
conda-build               3.19.2           py38h32f6830_4    conda-forge
conda-package-handling    1.7.0            py38h4d0b108_4    conda-forge
cryptography              3.0              py38h52adbb4_0    conda-forge
decorator                 4.4.2                      py_0    conda-forge
filelock                  3.0.12             pyh9f0ad1d_0    conda-forge
glob2                     0.7                        py_0    conda-forge
icu                       67.1                 hb1e8313_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
importlib-metadata        1.7.0            py38h32f6830_0    conda-forge
importlib_metadata        1.7.0                         0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
jinja2                    2.11.2             pyh9f0ad1d_0    conda-forge
jsonschema                3.2.0            py38h32f6830_1    conda-forge
jupyter_core              4.6.3            py38h32f6830_1    conda-forge
libarchive                3.3.3             hf12134e_1008    conda-forge
libcxx                    10.0.1               h5f48129_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libffi                    3.2.1             hb1e8313_1007    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
liblief                   0.10.1               hb1e8313_0    conda-forge
libxml2                   2.9.10               h2c6e4a5_2    conda-forge
lz4-c                     1.9.2                hb1e8313_3    conda-forge
lzo                       2.10              haf1e3a3_1000    conda-forge
markupsafe                1.1.1            py38h64e0658_1    conda-forge
nbformat                  5.0.7                      py_0    conda-forge
ncurses                   6.2                  hb1e8313_1    conda-forge
openssl                   1.1.1g               haf1e3a3_1    conda-forge
pip                       20.2.2                     py_0    conda-forge
pkginfo                   1.5.0.1                    py_0    conda-forge
psutil                    5.7.2            py38h4d0b108_0    conda-forge
py-lief                   0.10.1           py38hdfeb8a9_0    conda-forge
pycosat                   0.6.3           py38h64e0658_1004    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pyopenssl                 19.1.0                     py_1    conda-forge
pyrsistent                0.16.0           py38h64e0658_0    conda-forge
pysocks                   1.7.1            py38h32f6830_1    conda-forge
python                    3.8.5           hfc71d35_5_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python-libarchive-c       2.9                      py38_0    conda-forge
python.app                1.3              py38h64e0658_1    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2020.1             pyh9f0ad1d_0    conda-forge
pyyaml                    5.3.1            py38h64e0658_0    conda-forge
readline                  8.0                  h0678c8f_2    conda-forge
requests                  2.24.0             pyh9f0ad1d_0    conda-forge
ripgrep                   12.1.1               h0b31af3_0    conda-forge
ruamel_yaml               0.15.80         py38h64e0658_1001    conda-forge
setuptools                49.6.0           py38h32f6830_0    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
soupsieve                 2.0.1                      py_1    conda-forge
sqlite                    3.33.0               h960bd1c_0    conda-forge
tk                        8.6.10               hb0a8c7a_0    conda-forge
tqdm                      4.48.2             pyh9f0ad1d_0    conda-forge
traitlets                 4.3.3            py38h32f6830_1    conda-forge
urllib3                   1.25.10                    py_0    conda-forge
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
yaml                      0.2.5                haf1e3a3_0    conda-forge
zipp                      3.1.0                      py_0    conda-forge
zlib                      1.2.11                     1007    conda-forge
zstd                      1.4.5                h289c70a_2    conda-forge
$ conda config --add channels conda-forge
$ conda config --set channel_priority strict
$ conda update -n base conda conda-build

Note that the conda list did include python_abi, but I think that's ok, right? The update step also completed successfully this time. I then checked out a branch in the zstash repo to build the package.

$ conda build conda/ -c conda-forge
$ conda search --info --use-local zstash
Loading channels: done
zstash 0.4.2 py_0
-----------------
file name   : zstash-0.4.2-py_0.tar.bz2
name        : zstash
version     : 0.4.2
build       : py_0
build number: 0
size        : 17 KB
subdir      : noarch
url         : file:///opt/miniconda3/conda-bld/noarch/zstash-0.4.2-py_0.tar.bz2
md5         : 3ffae359bddc1ccf757dbf828c2b43c6
timestamp   : 2020-08-24 18:25:32 UTC
dependencies: 
  - python >=3.6

I don't see python_abi in that output. In that case, should I load this build to https://anaconda.org/e3sm/zstash and bump the build number as you suggest in #90?

xylar commented 4 years ago

Note that the conda list did include python_abi, but I think that's ok, right?

Yep, that's expected. python_abi is something to do with how conda-forge keeps track of different python flavors (CPython vs. PyPy) so it should be installed with every environment that includes python. But it should not be part of the dependencies of any noarch: python package (including zstash).

What you have looks good. But you already uploaded build 0 to the e3sm channel. So you need to merge my PR #90 first, then rebuild, then re-upload. By the way, anytime you build a package, it's safest to delete the miniconda3/conda-bld directory right before. In your case, /opt/miniconda3/conda-bld. That way, the build won't accidentally pick up other packages you've built in the past. I would suggest doing that before building build 1 even though it should be fine. That might prevent you from accidentally uploading build 0 instead of build 1.

forsyth2 commented 4 years ago

@xylar Ok I merged #90, then checked out a branch from the updated master and ran:

$ conda update -n base conda conda-build
$ rm -rf /opt/miniconda3/conda-bld
$ conda build conda/ -c conda-forge
$ conda search --info --use-local zstash # no python_abi in dependencies
$ anaconda upload /opt/miniconda3/conda-bld/noarch/zstash-0.4.2-py_1.tar.bz2 -u e3sm

https://anaconda.org/e3sm/zstash says that the last upload was a few seconds ago, so I believe this worked.

Let me know if this is sufficient. If so, I will update the "How to Prepare a Release" documentation (https://e3sm-project.github.io/zstash/docs/html/release.html#releasing-the-software-on-anaconda) accordingly. Thanks!

xylar commented 4 years ago

Looks good to me! I'm doing a trial build of e3sm-unified right now just to be sure.

xylar commented 4 years ago

Yep, we're good. I was able to include this in the e3sm-unified test build.