search

CMSCompOps / wtc-console

MIT License
1 stars 1 forks source link

readme

WTC-Console

This project uses Django React/Redux Base Project from Seedstarts as a boilerplate. Please refer to their github for the complete list of technologies used.

Here are the main tools whose knowledge is useful to contribute:

Frontend

Backend

Architecture

This app has a bit complicated architecture due to the nature of other systems it has to communicate.

Installing on a new machine (prod, dev)

Production/development setup uses nginx as reverse proxy and Gunicorn as an application server. Below are the steps needed to setup environment on RHEL from scratch. Instructions are based on this article How To Set Up Django with Postgres, Nginx, and Gunicorn on CentOS 7

Setting up environment

Use these instructions to setup a new production environment from scratch. By following these instructions you will create a dedicated user wtc-console for running this application with Gunicorn, update proxy config for Nginx and setup firewall to allow traffic on port 80.

Prerequisites:

Install Node and NPM

If you are using CERN managed machine then ask administrator to install latest Node version. If you are managing this machine, then follow this guide for RHEL

Install Oracle Instant Client

Install RabbitMQ

Add RabbitMQ repo according to RabbitMQ installation guide. Then run these commands:

Create user and login with it

Get the sources

Clone this project to wtc-console users home directory.

Setup oracle client

Open wtc-console users .bashrc file:

vim ~/.bashrc

Add these lines to it:

export ORACLE_HOME=/usr/lib/oracle/12.2/client64
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$PATH:$ORACLE_HOME/bin

And apply these changes:

. ~/.bashrc

Create virtual python environment

Install and configure nginx

Log out of wtc-console users session and with your user install nginx and dependencies

Add following lines to /etc/nginx/nginx.conf as a first server entry in http section and change domain_name to the actual domain name probably in format of _nodename.cern.ch

server {
    listen 80;
    server_name domain_name;
    location = /favicon.ico { access_log off; log_not_found off; }
    location / {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass http://0.0.0.0:8000;
    }
}

Test if configuration is valid

Start nginx and set it to run on startup

On RHEL and CentOS:

On Scientific Linux CERN:

For RHEL, Fedora, CentOS

If when opening domain.cern.ch you see this error in /var/log/nginx/error.log:

*2 connect() to 127.0.0.1:8000 failed (13: Permission denied) while connecting to upstream, client: some_ip, server: some_domain, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "some_domain"

Then use this command to solve it:

It turns on httpd connections and -P makes it persistent.

Give Nginx group rights to the project directory

Firewall config

Ask system administrators to include port 80 to puppet config. If puppet is not used, then you can configure firewall yourself to bypass traffic on this port with these commands:

You can see current config with sudo iptables --line -vnL or sudo less /etc/sysconfig/iptables

Update settings

For development node
For production node

Create prod.py in src/djangoreactredux/settings/ directory by using _prodtemplate.py settings template file and update the oracle and mongo db fields with prod values.

Create certificates for this machine and copy them to cert/ directory.

Next steps

Proceed to Development on dev machine or Production deployment steps depending on the machine purpose.

Development on remote machine

Development on remote node requires to run three sessions (terminals). First one will have frontent watch running, second will have backend running. And third one is for developent with editor of your choise.

Add your hostname to the list of allowed sites

Modify src/djangoreactredux/settings/base.py to include the assigned hostname:

ALLOWED_HOSTS = ['localhost','myDevTestMachine.cern.ch']
For all terminals

Become wtc-console user:

Go to project dir

In terminal 1

Start frontend resources watching:

In terminal 2

Start Django backend and Celery workers:

In terminal 3

Edit sources with an editor of you choise.

Certificates placement

To install CERN Certification Authority and User certificates:

mkdir src/djangoreactredux/cert && cd wtc-console/src/djangoreactredux/cert
wget -c "https://cafiles.cern.ch/cafiles/certificates/CERN%20Root%20Certification%20Authority%202.crt" -O CERNRootCertificationAuthority2.crt

And, depending on the location of your Key/Cert files:

cp ~/.globus/cert.pem crt.pem 
cp ~/.globus/key.pem .

The location of these files is specified via src/djangoreactredux/settings/dev.py

Production deployment

Become wtc-console user:

Go to project dir

Deployment is done with one bash command. It will:

./bin/deploy_prod.sh

Stopping server

If for some reason application should be stoppet then use this script:

./src/bin/stop_prod.sh

It will stop Gunicorn and Celery tasks

Maintenance

Logs are in /home/wtc-console/wtc-console/logs

Development guidelines

When developing a new feature create your own branch and push your changes at least daily.

Do not push directly to master. Create pull requests and assign someone to approve it. Go through your pull request your self, it helps to see if there is unwanted or commented-out code.

Adding new dependencies

While working on project you might encounter situations where you want to add functionality from third parties. This is done by adding dependecies to external libraries.

Frontend

Intall dependency with yarn

Backend

Install dependency with pypi