search

ESGF / esgf-installer

ESGF P2P Node Installer
https://esgf.llnl.gov/
Other
21 stars 21 forks source link

3.0 component upgrades must use specified versions #665

Open sashakames opened 5 years ago

sashakames commented 5 years ago

Apparently there are versions in esg_init.py that aren't considered in an upgrade. Also they may not be considered in installs from scratch but really should.

apache_frontend_version, apache_frontend_tag (needed for getting the esgf-httpd.conf template), but open to scrapping for 3.0

cdat_version # conda esgprep_version, # pip cmor_version, # conda esgf_dashboard_version, # dashboard_ip tarball or could be git repo esgf_stats_api_version, # java webapp - on mirror

The following are no longer deployed by the installer: esgf_desktop_version, openssl*, ant_version

nathanlcarlson commented 5 years ago

Regarding cdat, what exactly should be installed relating to cdat? This has been a very gray matter to me and it could use some specific definition.

sashakames commented 5 years ago

CDAT has a version check using "cdat_info", and upgrades have benefitted from uninstalling and reinstalling the following modules as follows:

conda remove -y cdms2 cdtime cdutil cdat_info libnetcdf conda install -y -c conda-forge cdms2 cdtime libnetcdf cdat_info

So far it hasn't been necessary to pin the cdat modules to specific versions, with the version number used for detection against the currently installed version, in order to suggest a needed upgrade

nathanlcarlson commented 5 years ago

Okay, but in the case of first time install what is the conda install command exactly for cdat module(s)? Or conda create as they may be included at creation time.

sashakames commented 5 years ago

first time it installs the modules within the "conda create": conda create -y -n esgf-pub -c conda-forge cdms2 cdtime cdat_info python=2.7 libnetcdf is left out because its a dependency. There have been issues with it not properly upgraded if not uninstalled and reinstalled, hence the need for that procedure in a upgrade. We are working around wonky/glitchy limitations of Conda.

nathanlcarlson commented 5 years ago

Okay I asked because those modules are currently not being specifically installed. Right now it looks like:

conda create -y -n esgf-pub "python<3" cdutil cmor "hdf5==1.10.3" -c pcmdi/label/nightly -c conda-forge

I am not sure if those modules are a subset of the dependencies here or what, but I would like to have a clear definition that I can safely follow. @sashakames Should it look like what I put above, or like what you put above

first time it installs the modules within the "conda create": conda create -y -n esgf-pub -c conda-forge cdms2 cdtime cdat_info python=2.7

sashakames commented 5 years ago

That looks somewhat outdated, due to divergence between the 2.x and 3.0 ongoing work. I assume this create step is in the bootstrap. cmor should be pinned to a version - we've had issues of new versions breaking.

nathanlcarlson commented 5 years ago

Okay, so not the nightly for cmor as previously discussed?

That looks somewhat outdated, due to divergence between the 2.x and 3.0 ongoing work.

Okay, is there a file maintained somewhere that can easily be referenced for what specifically should be installed?

sashakames commented 5 years ago

Nightly is a workaround. Whether we keep that depends on if it works vs just using the version in condo-forge. The code in "esg-node" is the reference for packages used in 2.x for conda/cdat.

nathanlcarlson commented 5 years ago

Looking more closely at cdat_version in the master branch I see that it actually is not used to specify the version of any module or package, it is only used to check the version installed. It appears that the latest versions of cdat related packages are being installed no matter what, which makes that check sort of artificial. Unless there is code some where external from the repository that uses it.