epics-services-deployment

A general guide to deploying EPICS services

EPICS Services Installation

Core Dependencies

sudo apt update && sudo apt upgrade
sudo apt install openjdk-17-jdk
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
sudo apt install git
sudo apt install maven
sudo apt install procserv

Create Services Account

adduser epics_services_nasa
usermod -aG sudo epics_services_nasa

ElasticSearch 8.2

Installation Guide: ElasticSearch 8.2

Download and Extract ElasticSearch 8.2

cd /tmp
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.2.3-linux-x86_64.tar.gz
mkdir -p /opt/epics-tools/services/nasa/elastic/
cd /opt/epics-tools/services/nasa/elastic
tar -xvf /tmp/elasticsearch-8.2.3-linux-x86_64.tar.gz

Configuration

Copy elasticsearch.yml to /opt/epics-tools/services/nasa/elastic/elasticsearch-8.2.3/config

Set Up ElasticSearch as a Service

Copy elastic.service to /etc/systemd/system/

systemctl enable elastic.service
systemctl restart elastic.service

MongoDB 7.0

Installation Guide: MongoDB on Debian 12

Install MongoDB 7.0

sudo apt-get install gnupg curl
curl -fsSL https://pgp.mongodb.com/server-7.0.asc | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/mongodb-server-7.0.gpg
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org

Change the data location in /etc/mongod.conf (ensure MongoDB user has write permissions).

Start and Enable MongoDB

sudo systemctl start mongod
sudo systemctl status mongod
sudo systemctl enable mongod

Kafka 3.6.2

Installation Guide: Kafka Documentation

Download and Extract Kafka

cd /tmp
wget https://downloads.apache.org/kafka/3.6.2/kafka_2.13-3.6.2.tgz
mkdir -p /opt/epics-tools/services/nasa/kafka
cd /opt/epics-tools/services/nasa/kafka
tar -xvf /tmp/kafka_2.13-3.6.2.tgz

Optional Configuration

Configure Zookeeper and Kafka ports, and set up the data folder in kafka/config/.

Set Up Kafka and Zookeeper Services

Copy zookeeper.service and kafka.service to /etc/systemd/system/.

systemctl enable zookeeper
systemctl enable kafka
systemctl start zookeeper
systemctl start kafka

Phoebus Installation

cd /tmp/
git clone https://github.com/ControlSystemStudio/phoebus.git && cd phoebus && git checkout d391b61 -b deploy
mvn clean install -DskipTests

Install Phoebus in your home directory

cp -rf /tmp/phoebus/phoebus-product/ ~

Copy phoebus-preferences.ini to ~/phoebus-product/.

Note the preferences are system specific and you will have to review and update n/w configurations, creadentails, alarm topics to match your deployment

./phoebus.sh -p phoebus-preferences.ini

Alarm Services

Alarm Server: Alarm Server
Alarm Logger: Alarm Logger
Alarm Config Logger: Alarm Config Logger

Install Alarm Server

mkdir -p /opt/epics-tools/services/nasa/phoebus-alarms
cp -rf /tmp/phoebus/services/alarm-server/target/service-alarm-server-4.7.4-SNAPSHOT-bin.tar.gz /opt/epics-tools/services/nasa/phoebus-alarms/
cd /opt/epics-tools/services/nasa/phoebus-alarms
tar -xvf service-alarm-server-4.7.4-SNAPSHOT-bin.tar.gz

Copy run-phoebus-alarm.sh to /opt/epics-tools/services/nasa/phoebus-alarms/.

Setup Alarm Preferences

Copy phoebus_alarm_preferences.ini to /opt/epics-tools/services/nasa/phoebus-alarms.

Setup Alarm Logger

mkdir /opt/epics-tools/services/nasa/phoebus-alarms/alarm_logger
cp -rf /tmp/phoebus/services/alarm-logger/target/service-alarm-logger-4.7.4-SNAPSHOT.jar /opt/epics-tools/services/nasa/phoebus-alarms/alarm_logger/

Copy the logger phoebus_alarm.properties to /opt/epics-tools/services/nasa/phoebus-alarms/alarm_logger/.
Copy run-phoebus-alarm-logger.sh to /opt/epics-tools/services/nasa/phoebus-alarms/alarm_logger/.

initialize the alarm configuration

Initialize the alarm topics using the create_alarm_topics scripts

Copy create_alarm_topics to /opt/epics-tools/services/nasa/phoebus-alarms.

./create_alarm_topics nasa_epics_alarm

Enable and Start Alarm Services

Setup the services using the template phoebus_alarm.services
Copy phoebus_alarm.service to /etc/systemd/system
Copy phoebus_alarm_logger.service to /etc/systemd/system

systemctl enable phoebus_alarm
systemctl start phoebus_alarm

systemctl enable phoebus_alarm_logger
systemctl start phoebus_alarm_logger

Verification

Start the Phoebus GUI: run-phoebus In the GUI menus, navigate to: Applications -> Alarm -> Alarm Tree and then repeat to Applications -> Alarm -> Alarm Log Table

Phoebus Olog

Installation Guide: Phoebus Olog

Download and Install

cd /tmp/
git clone https://github.com/Olog/phoebus-olog.git && cd phoebus-olog && git checkout f5022c4 -b deploy
cd phoebus-olog/
mvn clean install -Pdeployable-jar -DskipTests
mkdir -p /opt/epics-tools/services/nasa/olog
cp -rf /tmp/phoebus-olog/target/service-olog-5.0.0-SNAPSHOT.jar /opt/epics-tools/services/nasa/olog/service-olog-5.0.0.jar

Setup Startup Scripts and properties

Copy olog.properties to /opt/epics-tools/services/nasa/olog/.
Copy run-phoebus-olog.sh to /opt/epics-tools/services/nasa/olog/.

Enable and Start Olog Service

Copy phoebus_olog.service to /etc/systemd/system

systemctl enable phoebus_olog
systemctl start phoebus_olog

Check service status

curl http://localhost:9090/Olog

Follow the verification instructions on Github

Archiver Appliance

Install Archiver Appliance

Prerequisites

MySQL or MariaDB (with legacy encryption)
Gradle (for building the application)

Step 1: Build and Deploy the Archiver Appliance

Clone the Repository

cd /tmp/
git clone https://github.com/archiver-appliance/epicsarchiverap.git && cd epicsarchiverap && git checkout 75665cb -b deploy
cd /tmp/epicsarchiverap
./gradlew -x sphinx

or use pre built binaries https://github.com/archiver-appliance/epicsarchiverap/releases/tag/1.1.0

Prepare Directories

mkdir -p /tmp/epicsarchiverap-dep
cd /tmp/epicsarchiverap-dep

Step 2: Download Dependencies

Download Tomcat

wget https://archive.apache.org/dist/tomcat/tomcat-9/v9.0.31/bin/apache-tomcat-9.0.31.tar.gz

Download MySQL Connector

wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-j-8.2.0.tar.gz

Step 3: Setup RDB backend

Create the Archiver Database

See details for RDB Setup

sudo mariadb

In the database shell

CREATE DATABASE archappl;
CREATE USER archappl IDENTIFIED BY '******';
GRANT ALL ON archappl.* TO 'archappl'@'%' WITH GRANT OPTION;

Step 4: Install the Archiver Appliance

Prepare Installation Directory

mkdir -p /opt/epics-tools/services/nasa/aa/deploy
cp -rf /tmp/epicsarchiverap/build/distributions/archappl_v1.1.0-97-g75665cb.tar.gz /opt/epics-tools/services/nasa/aa/deploy/
cd /opt/epics-tools/services/nasa/aa/deploy
tar -xvf archappl_v1.1.0-97-g75665cb.tar.gz

Setup the MySQL Connector

cd /opt/epics-tools/services/nasa/aa/deploy
tar -xvf /tmp/epicsarchiverap-dep/mysql-connector-j-8.2.0.tar.gz

Create Installation Scripts Directory

mkdir -p /opt/epics-tools/services/nasa/aa/deploy/install_scripts

Copy Installation Scripts
Copy the following templated install scripts to install_scripts
- Copy single_machine_install.sh
- Copy addMysqlConnPool.py
- Copy deployMultipleTomcats.py

(Optional) Create Configuration Directory

mkdir -p /opt/epics-tools/services/nasa/aa/deploy/conf

Create/Verify the storage locations

Ensure the STS, MTS, and LTS locations exist.

sudo install -d -o epics_services_nasa -g epics_services_nasa /arch

sudo install -d -o epics_services_nasa -g epics_services_nasa /arch/sts/ArchiverStore
sudo install -d -o epics_services_nasa -g epics_services_nasa /arch/mts/ArchiverStore
sudo install -d -o epics_services_nasa -g epics_services_nasa /arch/lts/ArchiverStore
sudo mount -t tmpfs -o size=2048M tmpfs /arch/sts/ArchiverStore

Update the fstab file /etc/fstab

none /arch/sts/ArchiverStore tmpfs nodev,nosuid,noexec,nodiratime,size=2048M 0 0

Step 5: Run the Installation

Set Environment Variables

cd /opt/epics-tools/services/nasa/aa/deploy/install_scripts
export DEPLOY_DIR=/opt/epics-tools/services/nasa/aa/deploy
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
export TOMCAT_DISTRIBUTION=/tmp/epicsarchiverap-dep/apache-tomcat-9.0.31.tar.gz
export MYSQL_CLIENT_JAR=/opt/epics-tools/services/nasa/aa/deploy/mysql-connector-j-8.2.0/mysql-connector-j-8.2.0.jar
export MYSQL_CONNECTION_STRING="--host=localhost --user=archappl --password=***** --database=archappl"
# Optional
export SITE_SPECIFIC_POLICIES_FILE=/opt/epics-tools/services/nasa/aa/deploy/conf/policies.py

Execute the Installation Script
```
./single_machine_install.sh
```

Step 6: Configure Startup and Services

Setup Startup Scripts [IMPORTANT]
- Copy sampleStatup.sh to /opt/epics-tools/services/nasa/aa/deploy/nasaStartup.sh.
Setup Service Files
- Copy aa.service to /etc/systemd/system.

Modify HTML Management Pages

cd /opt/epics-tools/services/nasa/aa/deploy/mgmt/webapps/mgmt/ui/
find . -type f -name "*.html" -exec sed -i 's/LCLS/NSLS2/gI' {} +
find . -type f -name "*.html" -exec sed -i 's/Jingchen Zhou/AST/gI' {} +
find . -type f -name "*.html" -exec sed -i 's/Murali Shankar at 650 xxx xxxx or Bob Hall at 650 xxx xxxx/Kunal Shroff at shroffk@bnl.gov/gI' {} +

Switch Lab Logo
- Update logos in:
  - ../mgmt/webapps/mgmt/ui/comm/img/labLogo.png
  - ../mgmt/webapps/mgmt/ui/comm/img/labLogo2.png

Verify the service is running

Check the /opt/epics-tools/services/nasa/aa/deploy/mgmt/logs/arch.log

The following messages represent a good way to verify the AA services are running

[INFO ] 2024-09-24 16:43:51.705 [Startup executor] MgmtPostStartup - Finished post startup for the mgmt webapp
[INFO ] 2024-09-24 16:44:03.783 [http-nio-17665-exec-4] BasicDispatcher - Servicing /webAppReady
[INFO ] 2024-09-24 16:44:03.785 [http-nio-17665-exec-4] WebappReady - Received webAppReady from RETRIEVAL
[INFO ] 2024-09-24 16:44:03.802 [http-nio-17665-exec-5] BasicDispatcher - Servicing /webAppReady
[INFO ] 2024-09-24 16:44:03.803 [http-nio-17665-exec-5] WebappReady - Received webAppReady from ETL
[INFO ] 2024-09-24 16:44:04.200 [http-nio-17665-exec-6] BasicDispatcher - Servicing /webAppReady
[INFO ] 2024-09-24 16:44:04.201 [http-nio-17665-exec-6] WebappReady - Received webAppReady from ENGINE
[INFO ] 2024-09-24 16:44:06.199 [Startup executor] MgmtPostStartup - About to run MgmtPostStartup
[INFO ] 2024-09-24 16:44:06.200 [Startup executor] MgmtPostStartup - Startup is complete for MgmtPostStartup
[INFO ] 2024-09-24 16:44:11.070 [http-nio-17665-exec-4] MgmtRuntimeState - All components in this appliance have started up. We should be ready to start accepting UI requests

ChannelFinder

Installation Guide: ChannelFinder Service

Install ChannelFinder

cd /tmp/
git clone https://github.com/ChannelFinder/ChannelFinderService.git  && cd ChannelFinderService && git checkout 3a3c94f -b deploy

cd ChannelFinderService
mvn clean install -DskipTests
mkdir -p /opt/epics-tools/services/nasa/cf
cp -rf /tmp/ChannelFinderService/target/ChannelFinder-4.7.3-SNAPSHOT.jar /opt/epics-tools/services/nasa/cf/ChannelFinder-4.7.3.jar

Copy cf.properties to /opt/epics-tools/services/nasa/cf/.
Copy run-cf.sh to /opt/epics-tools/services/nasa/cf/.

Enable and Start ChannelFinder Service

Copy cf.service to /etc/systemd/system.

systemctl enable cf
systemctl start cf

Check service status

curl http://localhost:7171/ChannelFinder

Follow the verification instructions on Github

Setting up recsync

sudo apt install python3-twisted python3-requests python3-simplejson python3-urllib3

cd /tmp
git clone https://github.com/ChannelFinder/RecSync-env && cd RecSync-env && git checkout b01bd93 -b deploy
cd RecSync-env

make init
make install

# in /etc/systemd/system/recsync.service
# change channelfinder.service to cf.service
# change /usr/bin/twistd to /usr/bin/twistd3

systemctl daemon-reload
systemctl start recsync.service
systemctl status recsync.service

Recsync configuration

In /opt/RecSync/recsync.conf set addrList

In /opt/RecSync/channelfinderapi.conf set the CF url, username, and password

Save/Restore

mkdir -p /opt/epics-tools/services/nasa/save_restore
cp /tmp/phoebus/services/save-and-restore/target/service-save-and-restore-4.7.4-SNAPSHOT.jar /opt/epics-tools/services/nasa/save_restore/service-save-and-restore-4.7.4-SNAPSHOT.jar

Copy save_restore.properties to /opt/epics-tools/services/nasa/save_restore/.
Copy run-save-restore.sh to /opt/epics-tools/services/nasa/save_restore/.

Enable and Start Save/Restore Service

Setup services using save_restore.service as a template.
Copy save_restore.service to /etc/systemd/system.

systemctl enable save_restore
systemctl start save_restore

Check service status

curl http://localhost:6060/save-restore

Follow the verification instructions on Github

osprey-dcs / epics-services-deployment

readme

epics-services-deployment

EPICS Services Installation

Core Dependencies

Create Services Account

ElasticSearch 8.2

Download and Extract ElasticSearch 8.2

Configuration

Set Up ElasticSearch as a Service

MongoDB 7.0

Install MongoDB 7.0

Start and Enable MongoDB

Kafka 3.6.2

Download and Extract Kafka

Optional Configuration

Set Up Kafka and Zookeeper Services

Phoebus Installation

Alarm Services

Install Alarm Server

Setup Alarm Preferences

Setup Alarm Logger

initialize the alarm configuration

Enable and Start Alarm Services

Verification

Phoebus Olog

Download and Install

Enable and Start Olog Service

Check service status

Archiver Appliance

Install Archiver Appliance

Prerequisites

Step 1: Build and Deploy the Archiver Appliance

Step 2: Download Dependencies

Step 3: Setup RDB backend

Step 4: Install the Archiver Appliance

Step 5: Run the Installation

Step 6: Configure Startup and Services

Verify the service is running

ChannelFinder

Install ChannelFinder

Enable and Start ChannelFinder Service

Check service status

Setting up recsync

Recsync configuration

Save/Restore

Enable and Start Save/Restore Service

Check service status