New DI website 2019
Pattern library imported from https://gitlab.com/the-b-team/development-initiatives-website/ on 12.6.2019.
Pattern library available online at http://development-initiatives.surge.sh/
cd patterns
nvm install 6.13.0
nvm exec 6.13.0 npm i
nvm exec 6.13.0 npm run build
Create a local database
psql -U postgres -c 'CREATE DATABASE devinit;'
Create a virtual environment
python3 -m virtualenv venv
Source your virtual environment
source venv/bin/activate
Add database URL to the .env
file. This shall be read by django-dotenv
echo "DATABASE_URL=postgresql://di_website:di_website_pw@db/di_website" > .env
Install python dependencies
pip install -r requirements.txt
Install node dependencies
npm install
Build JS
npm run build
Migrate
python3 manage.py migrate
Add
export DJANGO_SETTINGS_MODULE=di_website.settings.dev
to your activate file inside
path_to_virtual_env_folder/bin/
Build static assets - refer to Pattern library section, then in the project root
python3 manage.py collectstatic
Run
python3 manage.py runserver
Test
python3 manage.py test
Install dependencies
Run docker-compose
with the docker-compose.minimal.yml
file, as below
docker-compose -f docker-compose.minimal.yml up --build
Steps to develop with docker:
Create docker volume diwebsite_db
docker volume create --name=diwebsite_db
Run command:
docker-compose -f docker-compose.dev.yml up --build
You'll need to manually run migrations:
docker-compose exec web python manage.py migrate
If you wish to test the production build, run:
docker-compose up --build -d
If the deployment is from scratch, follow commands below to update content with old website content
Execute the commands below to finish the migration
docker-compose exec web python manage.py fixblogs
docker-compose exec web python manage.py importwp
docker-compose exec web python manage.py update_index
Fetch data from the old CMS repo
python3 manage.py fetch_spotlight_data
Process downloaded files and import data into your database
python3 manage.py import_spotlight_data
If you'd added Spotlight data in one environment and wish to import it into the current one
python3 manage.py update_spotlights_from_api base_url=[Specify Source URL]
NB: default base_url is http://178.128.102.213/
The development environment has been setup using grunt, webpack & typescript.
Widget code sits in the src
directory. Update the webpack.config.js
with your widget's configuration then run:
npm run dev
To bundle your code, run:
npm run build
# Upgrade general dependencies
apt update
apt upgrade
# Install docker
sudo snap refresh && sudo snap install docker
# Install docker ce and compose
apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
"deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
"$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
# Start docker daemon
sudo systemctl enable docker.service
sudo systemctl enable containerd.service
# Enable docker logrotate
nano /etc/docker/daemon.json
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}
systemctl restart docker.service
# Turn on firewall
ufw allow 80
ufw allow 443
ufw allow 22
ufw enable
# Create user and add to docker group
adduser di_website
groupadd docker
usermod -aG docker di_website
# Install npm
apt install npm
# Set up containers
su di_website
cd ~
git clone https://github.com/devinit/DIWebsite-redesign.git
APP_NAME="DIwebsite-redesign"
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
APP_DIR=$SCRIPT_DIR'/'$APP_NAME
DATABASE_BACKUP=$SCRIPT_DIR'/database_backup'
DATABASE_NAME='di_website'
DOCKER_STORAGE='diwebsite_db;index_db'
REPOSITORY="git@github.com:devinit/"$APP_NAME".git"
cat "$APP_DIR/config/nginx/django_https.ctmpl" >> $APP_DIR"/config/nginx/django.conf.ctmpl"
cd ~/DIWebsite-redesign
chmod +x scripts/*
cd ssl
openssl genrsa > privkey.pem
openssl req -new -x509 -key privkey.pem > fullchain.pem
cd ..
docker pull gliderlabs/registrator:latest
docker network create "consul"
docker-compose -f docker-compose-consul.yml up -d
new_state='blue'
docker build . -t diwebsite-redesign_web:new
docker tag diwebsite-redesign_web:new diwebsite-redesign_web:${new_state}
state='green'
docker tag diwebsite-redesign_web:new diwebsite-redesign_web:${state}
docker volume create --name=diwebsite_db
docker-compose up -d ${new_state}
docker-compose exec -T db psql
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
docker-compose exec -T db psql -U di_website -d di_website < ../2023-08-09.backup
docker-compose down
source scripts/init.sh
docker-compose stop green
docker-compose exec -T ${new_state} python manage.py update_index
new_upstream=${blue_upstream}
./scripts/activate.sh ${new_state} ${state} ${new_upstream} ${key_value_store}
docker-compose restart nginx
If you accidentally have multiple IP addresses for one service in Consul (e.g. stopped service while consul was down)
Check nginx:
docker-compose exec nginx cat /etc/nginx/conf.d/nginx.conf
Find real IP address:
docker container ps
docker inspect e20e0a57d399
"IPAddress": "192.168.208.6",
First, bash into nginx, then curl -sX GET http://consul:8500/v1/catalog/service/blue
to get the ServiceId. Followed by docker-compose -f docker-compose-consul.yml exec consul consul services deregister -id=ab9298cc69bd:blue:8090
We are using wagtail-2fa. The documentation about which version of this package matching with a specific version of wagtail is terrible. You will have to go through all the changelogs or something similar. If you intend to upgrade either Wagtail
or wagtail-2fa
, we advise you set DEBUG = False
after running all your tests in debug mode, and test 2FA login as well with debug set off. This is because this package attempts to use wagtail.svg
which keeps changing names in the Wagtail package, and ManifestStaticFilesStorage
will not throw any errors with DEBUG = True
. So the only way to test that the two packages (Wagtail and wagtail-2fa) are compatible is turning off debug. Also, don't forget to collectstatic
before testing.