SURF SamenwerkingsBeheerSysteem (SBS)

SURF Research Access Management (SRAM) Platform

Overview Requirements

Python 3.9.x
MySQL v5.7.x or MariaDB 10.x
Redis v6.x
Yarn 1.x
node
libxmlsec1 (pre-1.3.0 see https://github.com/xmlsec/python-xmlsec/issues/254 and see https://github.com/xmlsec/python-xmlsec/issues/254#issuecomment-1511135314 for a workaround)

Getting started

Server

Create a virtual environment and install the required python packages:

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r ./server/requirements/test.txt

Connect to your local mysql database: mysql -uroot and create the SBS database and user:

DROP DATABASE IF EXISTS sbs;
DROP DATABASE IF EXISTS sbs_test;
CREATE DATABASE sbs CHARACTER SET utf8mb4 DEFAULT CHARACTER SET utf8mb4;
CREATE DATABASE sbs_test CHARACTER SET utf8mb4 DEFAULT CHARACTER SET utf8mb4;
CREATE USER 'sbs'@'localhost' IDENTIFIED BY 'sbs';
GRANT ALL PRIVILEGES ON *.* TO 'sbs'@'localhost' WITH GRANT OPTION;

Ensure MySQL is running and run the Python server with the correct local environment settings:

PROFILE=local ALLOW_MOCK_USER_API=1 CONFIG=config/test_config.yml python -m server

With TESTING=1 no mails will be sent. If you do want to validate the mails you can run a fake smtp server with:

python -m smtpd -n -c DebuggingServer localhost:1025

If you want the emails to be opened in the browser when developing add the OPEN_MAIL_IN_BROWSER=1 to your environment. Or even better, use https://mailpit.axllent.org/ and capture all emails send. You can see all mails delivered at http://0.0.0.0:8025/ when you have the following configuration in you config file:

mail:
  host: 0.0.0.0
  port: 1025

Client

First install all dependencies with:

yarn install

The GUI can be started with:

cd client
yarn start

To create a GUI production build:

yarn build

To analyze the bundle:

yarn analyze

API

See the Swagger for the API documentation and data model. Or for local development http://localhost:8080/apidocs/

You can use the Swagger interface for testing or cUrl using the command line:

curl -H "Accept: application/json" -H "Content-type: application/json" -H "Authorization: bearer {api_key}" "http://localhost:8080/api/organisations/v1" | jq .
curl -H "Accept: application/json" -H "Content-type: application/json" -H "Authorization: bearer {api_key}" "http://localhost:8080/api/collaborations/v1/{co_identifier}" | jq .

Routes

To see all routes:

source .venv/bin/activate
cd server
CONFIG='config/test_config.yml' FLASK_APP='__main__.py' flask routes

Testing

To run the tests, you need a functioning Redis server on localhost:6379.

To run all Python tests and validate syntax / formatting:

source .venv/bin/activate
cd server
pytest test
flake8 ./

To generate the coverage reports:

source .venv/bin/activate
cd server
pytest --cov=server --cov-report html:htmlcov test
open htmlcov/index.html

Within PyCharm you must mark the SBS/server/test directory as Test sources root in order to execute conftest.py before tests are run. See https://intellij-support.jetbrains.com/hc/en-us/community/posts/12897247432338-PyCharm-unable-to-find-fixtures-in-conftest-py

If you are getting errors in Pycharm when debugging, then have a look at https://youtrack.jetbrains.com/issue/PY-51495/PyCharm-debug-fails-upon-import-asyncio

To run all JavaScript tests:

cd client
yarn test

Or to run all the tests and do not watch - like CI:

cd client
CI=true yarn test

With the environment variable CONFIG=config/test_config.yml the test database is used. After you ran one or all of the tests the database is left with the test data seed. If you want to skip the login process when developing local then add the following to your environment: OPEN_MAIL_IN_BROWSER=1;PROFILE=local;CONFIG=config/test_config.yml;ALLOW_MOCK_USER_API=1

Deployment

See the https://github.com/SURFscz/SCZ-deploy project

Upgrade

See https://github.com/simion/pip-upgrader for upgrading automatically

source .venv/bin/activate
pip install pip-upgrader
cd server
pip-upgrade requirements/test.txt --dry-run

SURFSecureID

See the /config/saml_test configuration and the https://github.com/SURFscz/SCZ-deploy project

flask

To open a Flask terminal session:

source .venv/bin/activate
cd server
CONFIG='config/test_config.yml' FLASK_APP='__main__.py' flask shell

PyCharm

Because of eventlet the debugger in PyCharm sometimes crashes. Use the following environment properties to resolve this:

ALLOW_MOCK_USER_API=1;
CONFIG=config/acc_config.yml;
EVENTLET_HUB=poll;
PROFILE=local;
PYDEVD_USE_CYTHON=NO;
PYDEVD_USE_FRAME_EVAL=NO;
PYTHONUNBUFFERED=1

docker

For localhost deployment you can make use of docker. You can take existing docker-compose.yml as your starting point. The docker-compose file makes use of environment variables that you can adjust via a local .env file. To create your own .env file, copy the provided .env.example file:

$ cp .env.example .env

Now adjust the contents of this .env file to match your desired configuration.

Then build the docker images and launch the containers:

$ docker-compose build
$ docker-compose up -d

Now open your browser at: http://localhost:8080

SURFscz / SBS

readme