conda problems during upgrade

[alaniwi]

Tested upgrade from 2.6.7 to 2.7.1

Somewhere during the update it seems to be conda-installing some packages but into the root conda environment and others into the esgf-pub one. When it tries to run esginitialize during the installation, it finds the one in the root environment, which requires cdtime. This is missing from the root environment, so this falls over and aborts the installer. Meanwhile, it has installed cdtime into the esgf-pub conda environment. Frustrating because we are not even trying to run the publisher on the servers -- we have a separate publisher node for that!

[alaniwi]

After the aborted update, the root envrionment contains these:

root@esgf-test2:~# /usr/local/conda/bin/conda list  | egrep -i 'cmor|esg|cdtime'
esgcet                    3.5.3                     <pip>
ESGConfigParser           0.1.17                    <pip>
esgf-dashboard            0.0.2                     <pip>
esgf-pyclient             0.2.1                     <pip>
esgfpid                   0.7.11                    <pip>
esgprep                   2.9.2                     <pip>

and the esgf-pub environment contains these:

root@esgf-test2:~# /usr/local/conda/bin/conda list -n esgf-pub | egrep -i 'cmor|esg|cdtime'
# packages in environment at /usr/local/conda/envs/esgf-pub:
cdtime                    3.0              py27h6091dcd_1    conda-forge
esgcet                    3.4.2                     <pip>
ESGConfigParser           0.1.17                    <pip>
esgf-dashboard            0.0.2                     <pip>
esgf-node-manager         0.1.5                     <pip>
esgf-pyclient             0.1.8                 py27_1001    conda-forge
esgfpid                   0.7.11                    <pip>
esgprep                   2.7.38                    <pip>

For now I will probably just work around it by explicitly installing cdtime into the root conda environment before starting the update. As I am not trying to run the publisher on the servers, I don't really care about what goes where, provided that it doesn't abort the update. But probably there is just an activate statement missing somewhere in the installer.

[sashakames]

Looks like pip installed packages end up in both environments, but we use "conda install" for those deployed only via conda-forge. The latter include the CDAT packages which are dependencies of the publisher.

[alaniwi]

Seems that the versions in esgf-pub are what was installed during the 2.6.7 installation. The versions in root are the attempted update.