search

ranjanistic / knotters

The Knotters platform.
Other
3 stars 0 forks source link
django

readme

Knotters

The Knotters Platform source code.

See CONTRIBUTING.md if you have read the whole README.md and have already setup the repository as instructed in this file. If not, then go through the following steps first, then jump to CONTRIBUTING.md for further contribution guidelines.

Join the internal communications channel on our own Knotters Discord Server for discussions.

Setup

Linux environment is preferred, so if you were planning to boot/dual boot your pc with linux, this is the right time.

For windows developers who don't want to boot linux, setup WSL first.

All commands/bash scripts should be assumed to be executed in the root of project directory, unless specified explicitly.

python3 (python), pip (python package manager) - these cmdlets may vary depending on your system platform, therefore, act accordingly.

Prerequisites

For setting up mongodb first time locally, check these steps

For setting up redis first time locally, check these steps

For WSL users, commands like systemctl may not be present, therefore proceed accordingly. (E.g. use service command for systemctl)

After setting up redis, add a user&password pair in acl section in your redis configuration file (typically at /etc/redis/redis.conf), by adding the following line in it

user <username> on ><password> allkeys allcommands

Restart redis after any changes to its configuration.

Make sure mongodb is running, using

mongosh "mongodb://localhost:27017/"
...
test>

Make sure redis is running using

redis-cli
127.0.0.1:6379> auth <user> <password>

Environment

cd ~
mkdir Knotters
cd ~/Knotters
git clone <repo-clone-url>
cd ~/Knotters/knotters
python3 setup.py

Check values in main/.env and main/.env.testing manually as well for everything is ok or not.

Dependencies

pip install wheel

Create a python virtual environment

cd ~/Knotters
python3 -m venv knottersenv

Activate it

source ~/Knotters/knottersenv/bin/activate

Then install dependencies

cd ~/Knotters/knotters
pip install -r requirements.txt

Static setup

Set STATIC_ROOT in main/.env and main/.env.testing (both should have same values for this) to the absolute path of an empty directory (like /var/www/knotters/static/) where you can allow server to load static files from the static folder. Make sure whichever path you set is not restricted for server by any directory permissions. Also make sure that you DO NOT set the STATIC_ROOT as path to the ./static folder in this project's directory in any way.

For example, if you set STATIC_ROOT as /var/www/knotters/static/, then run

sudo mkdir -p /var/www/knotters/static/
sudo chown -R <user>:<user> /var/www/knotters/

Then, use the following to load static files.

python3 genversion.py
python3 manage.py collectstatic

This will prevent tests from failing which depend on the presence of static files at your STATIC_ROOT location.

You can also run the follwing to compress the collected static files at your STATIC_ROOT, although it is not neccessary for development/testing process.

python3 manage.py preparestatics /var/www/knotters/errors/

Database Setup

python3 manage.py makemigrations
python3 manage.py migrate

Initial setup

Create default necessary records

python3 manage.py setup

Notification setup

Synchronise your database with notifications data from the source code

python3 manage.py syncnotifications

Accounts setup

Create a superuser account for yourself.

python3 manage.py createsuperuser

This account can be used to login and access the administration view as well, at http://localhost:8000/<ADMINPATH>/. The <ADMINPATH> here is an environment variable from your .env.

Server

Change your working branch (always create any new branch from branch:beta)

git checkout beta
git pull
git checkout -b <your-branch-name>
git status

Activate environment if not activated

source ~/Knotters/knottersenv/bin/activate

Then run the main server process (assuming port 8000 is free)

python3 manage.py runserver

You're now eligible to jump into CONTRIBUTING.md, but it is recommended only after reading the complete README.md first.

Testing

Make sure that main/.env.testing is set appropriately.

NOTE: BEFORE RUNNING TESTS, PLEASE REMOVE ALL REDIS RELATED VALUES FROM YOUR main/.env.testing FILE.

python3 manage.py test
python3 manage.py test <module-name>
python3 manage.py test --tag=<tagname>
python3 manage.py test --parallel --verbosity=2

For coverage report of tests

coverage run --source='.' manage.py test
coverage report
coverage html

Optionally

The following setups are not neccessary for the server to run, but might be required at some point, so it is recommended to go through the following too.

Cluster setup

A qcluster can be started in a separate shell process, which runs in parallel with the main server process to handle time consuming tasks.

# requires redis config in .env file
python3 manage.py qcluster

Client setup

If you want to have control over client side service worker updates, the following command will create/update a version tag at main/__version__.py on every execution, which is linked directly with the service worker, forcing it to emit an update via web browser.

python3 genversion.py

Static Libraries Update

This should only be used when static libraries need to be updated. If updated, then re-check if everything from the updated libraries work fine on client side.

python3 static.py

This will read libraries source paths mentioned in static.json, and overwrite the library files present. You should also add the update source for any new client side libraries you may use, in this file.

Language setup

Set locale destination path in your main/.env file as LOCALE_ABS_PATH.

You can clone the existing Knotters translation repository from github.com/Knotters/knottrans, and set the LOCALE_ABS_PATH as /<local-path-to-clone>/knottrans/locale/, after cloning and create a new branch in it, before generating/updating .po files in it.

To generate/update the .po files for translations

python3 manage.py makemessages --extension=html,js

To compile .po files to .mo files present at your LOCALE_ABS_PATH.

python3 manage.py compilemessages

If you are using the Knotters translation repository for translations, then create your own branch in that repository too, and commit & push the changes as well. To actually deploy your new translations, create a pull request in that repository only after your related changes here in this repository have been successfully deployed on branch:main.

✨ If you have signed up on Knotters platform and using the same GitHub account linked with it and committing in Knotters translation repository, then contributing in our translations repository provides you XPs on Knotters platform too, as it is a part of a verified project on knotters.org itself! ✨

Deployment

All workflows are kept in .github/workflows/ directory.

There are total 4 workflows in the repository, for different event trigger cases.

--

--

There are total 5 runners in the repository, hosted on our own servers, for the above workflows do their jobs. Following list explains the job of each runner.

--

--

Footnotes

Jump to CONTRIBUTING.md for detailed contribution guide now.