DEPLOY: Release to instruments

John-Holt-Tessella commented 4 years ago

Release latest version of IBEX to instruments below

TODO: ticket #5060 should add list of instruments here* Release to all instruments

Acceptance Criteria

Ensure that version numbers on IBEX page is up to date

Tom-Willemsen commented 4 years ago

ZOOM and IMAT have nagios alarms on disk space at the moment. It would be worth checking that these issues are resolved after deploying (it is likely to simply be large databases from last cycle).

ThomasLohnert commented 4 years ago

Full Deployments

ALF

[x] Started
[x] Completed

CRISP

[x] Started
[x] Completed

DETMON (impossible to do for now, DETMON cant access shares drive even with the credentials added)

[ ] Started
[ ] Completed

EMMA

[x] Started
[x] Completed

EMU (Deploy server 6.0.1 with client 6.0.0 and python 6.0.0)

[x] Started
[ ] Completed

ENGINX

[x] Started
[x] Completed

ENGINX_SETUP

[x] Started
[x] Completed

GEM

[x] Started
[x] Completed

HRPD

[x] Started
[x] Completed

IMAT

[x] Started
[x] Completed

INES

[x] Started
[x] Completed

IRIS (Before Tuesday)

[x] Started
[x] Completed

IRIS_SETUP (Before Tuesday)

[x] Started
[x] Completed

LARMOR

[x] Started
[x] Completed

LET

[x] Started
[x] Completed

LOQ

[x] Started
[x] Completed

MAPS

[x] Started
[x] Completed

MARI

[x] Started
[x] Completed

MERLIN

[x] Started
[x] Completed

MUONFE

[x] Started
[x] Completed

NIMROD

[x] Started
[x] Completed

OSIRIS

[x] Started
[x] Completed

POLARIS

[x] Started
[x] Completed

RIKENFE

[ ] Started
[ ] Completed

SANDALS

[x] Started
[x] Completed

SELAB (do not do before 11/02/2020, ask JN before doing)

[ ] Started
[ ] Completed

SOFTMAT

[x] Started
[x] Completed

TOSCA

[x] Started
[x] Completed

ZOOM (Check they are done setting up sample stack)

[x] Started
[x] Completed

Deploy Mini-Inst

INTER

[x] Started
[x] Completed

MUSR

[x] Started
[x] Completed

OFFSPEC

[x] Started
[x] Completed

POLREF

[x] Started
[x] Completed

SANS2D

[x] Started
[x] Completed

SURF

[x] Started
[x] Completed

SXD

[x] Started
[x] Completed

Configure IE only

ARGUS

[x] Started
[x] Completed

CHIPIR

[x] Started
[x] Completed

CHRONUS

[x] Started
[x] Completed

HIFI

[x] Started
[x] Completed

PEARL

[x] Started
[x] Completed

VESUVIO

[x] Started
[x] Completed

WISH

[x] Started
[x] Completed

ThomasLohnert commented 4 years ago

Issues encountered during running the upgrade script:

Deleting backups fails

The upgrade script often fails to remove some of the backup folders. The script will tell you this - just delete it manually, then continue with the script

In some instances, the script fails to delete the current EPICS folder after making a backup because ca.dll is still in use because for some reason caRepeater is still running after the script calls stop_ibex_server. In these cases, open task manager, stop caRepeater and delete EPICS.

MYSQL install fails

Sometimes, the script will fall over while updating to the latest version of mysql. It can then happen that when you retry the task, the script thinks it already is on the latest version and skips the upgrade step.

To work around this, you need to edit the install script and comment out the bit that skips the upgrade:

Copy <share>\ibex_utils\installation_and_upgrade somewhere locally onto the NDX machine

in \installation_and_upgrade\install_utils\install_tasks.py, in the method install_mysql, comment out the following lines:

        if MYSQL_LATEST_VERSION in version and not force:
            print("MySQL already on latest version ({}) - nothing to do.".format(MYSQL_LATEST_VERSION))
            return

Re-run the deploy script in the local directory (you can skip all the steps you have previously completed)

Java home not found

If you repeatedly get popups that java home has not been found, that means the remote desktop session has not picked up new environment variables after installing the latest java. Stopping the server (if running) and reconnecting with remote deskop should fix this issue.

IBEX Server does not automatically start

After the deploy is finished (after the step "Reapply Hotfixes"), the script should automatically start up the IBEX server before going into the last section of manual server/client tests. This does not work properly, you will have to run start_ibex_server.bat before you carry on with the release tests.

aaron-long commented 4 years ago

Instruments using LINMOT motor controllers require the hotfix [#5163].

Without the hotfix the controller will not correctly initiate and motor records will go into major alarm.

JamesKingWork commented 4 years ago

Any reason why the following haven't been updated? IRIS_SETUP, VESUVIO, RIKENFE, POLARIS, OSIRIS, INES, NIMROD, SURF, SXD, POLREF, SANS2D

Tom-Willemsen commented 4 years ago

Polaris, Ines, Nimrod and Osiris all look updated according to the version PV. How are you checking them?

c:\Instrument\Apps\tools>caget IN:POLARIS:CS:VERSION:SVN:REV IN:OSIRIS:CS:VERSION:SVN:REV IN:NIMROD:CS:VERSION:SVN:REV IN:INES:CS:VERSION:SVN:REV
IN:POLARIS:CS:VERSION:SVN:REV  6.0.0.465df1d
IN:OSIRIS:CS:VERSION:SVN:REV   6.0.0.465df1d
IN:NIMROD:CS:VERSION:SVN:REV   6.0.0.465df1d
IN:INES:CS:VERSION:SVN:REV     6.0.0.465df1d

JamesKingWork commented 4 years ago

Polaris, Ines, Nimrod and Osiris all look updated according to the version PV. How are you checking them?

c:\Instrument\Apps\tools>caget IN:POLARIS:CS:VERSION:SVN:REV IN:OSIRIS:CS:VERSION:SVN:REV IN:NIMROD:CS:VERSION:SVN:REV IN:INES:CS:VERSION:SVN:REV
IN:POLARIS:CS:VERSION:SVN:REV  6.0.0.465df1d
IN:OSIRIS:CS:VERSION:SVN:REV   6.0.0.465df1d
IN:NIMROD:CS:VERSION:SVN:REV   6.0.0.465df1d
IN:INES:CS:VERSION:SVN:REV     6.0.0.465df1d

I was just checking the instrument columns https://github.com/ISISComputingGroup/IBEX/wiki#instrument-information. I will update that info though

JamesKingWork commented 4 years ago

ZOOM and IMAT have nagios alarms on disk space at the moment. It would be worth checking that these issues are resolved after deploying (it is likely to simply be large databases from last cycle).

There are no longer Nagios warnings on disk space.

JamesKingWork commented 4 years ago

Any reason RIKENFE and SELAB haven't been deployed to

C:\Instrument\Apps\EPICS>caget IN:SELAB:CS:VERSION:SVN:REV IN:RIKENFE:CS:VERSION:SVN:REV
IN:SELAB:CS:VERSION:SVN:REV    5.5.0.1e45035
IN:RIKENFE:CS:VERSION:SVN:REV  5.5.0.1e45035

Tom-Willemsen commented 4 years ago

RIKENFE couldn't be deployed to as they were testing PLC code (which required use of IBEX) at the time when we wanted to deploy. SELAB I think was in use at deployent time - but it is not a "production" instrument so we may be able to update it now if we ask electronics group nicely

ISISComputingGroup / IBEX