Setting up CI pipeline with GitHub actions

[han16nah]

Set up GitHub action.

When to run workflow

Configure the action to be checked every time someone pushes to main or issues a pull request to main. Thus, the yml file should start with:

name: Python application

# Controls when the action will run. 
on:
  # Triggers the workflow on push or pull request events but only for the main branch
  push:
    branches: [ main ]
  pull_request:
    branches: [ main]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

Note the workflow_dispatch option, which allows to also trigger the workflow manually.

Building the application

Next step is building the application. For this we, have to decide for the operating systems and the Python versions, e.g. latest Version of Windows, Ubuntu and Mac OS and Python versions 3.7, 3.8, 3.9 and 3.10 like below. Furthermore, dependencies have to be installed, e.g. from a requirement.txt:

jobs:
  build:

    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: true
      matrix:
        os: ["ubuntu-latest", "windows-latest", "macos-latest"]
        python-version: ["3.7", "3.8", "3.9", "3.10"]

    steps:
    - name: Checkout repository
      uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt

Requirements can also be installed using anaconda/miniconda, but this is more complicated and works differently for different OS. I would therefore opt for creating a requirements.txt in addition to our environment.yml. Source: https://autobencoder.com/2020-08-24-conda-actions/

Important note: Start with a minimal example (e.g., only "windows-latest" and Python version "3.9") and add other OS and Python versions after everything works successfully for one version (other wise workflow runs take very long and it's more difficult to find out where the issue is).

Running pytests

For this, we have to create repository-level GitHub secrets, which hold the mongoDB credentials. These variables can then be taken from these secrets before executing the pytests.

    - name: Test with pytest
      env:
        CONN_URI: ${{ secrets.CONN_URI }}
        CONN_DB: ${{ secrets.CONN_DB }}
        CONN_COL: ${{ secrets.CONN_COL }}
      run: |
        python -m pytest --cov pytreedb

see also: https://docs.github.com/en/actions/security-guides/encrypted-secrets

Using the following lines in the python test files (as is the case at the moment), the credentials should (hopefully) automatically be loaded from what is set as environment variables in the workflow before executing the pytests.

load_dotenv()
conn_uri = os.environ.get("CONN_URI")
conn_db = os.environ.get("CONN_DB")
conn_col = os.environ.get("CONN_COL")

Optional: Running linter

    - name: Run linter
      run: |
        flake8
        flake8_nb

Optional: Building documentation

    - name: build sphinx doc
      run: |
        cd doc
        make html

PS: I was just trying to provide a guide here, it's pretty likely that some parts of the code have to be changed for it to work 🙈