Python Driver Matrix

Prerequisites

• Python3.8
• pip
• docker
• git
• OpenJDK 8

Installing dependencies

Following commands will install all project dependencies using Pipenv

• Updates the package sources list

  sudo apt-get -y update

• git install.

  sudo apt-get install -y git-all

• pip install

  sudo apt-get install -y python3-pip

• python imported packages (Please note that the default Python version should be 3.8)

  apt-get install -y build-essential libssl-dev libffi-dev python3-dev python3.8-venv
  

Repositories dependencies

All repositories should be under the same base folder

  git clone git@github.com:datastax/python-driver.git datastax-python-driver &
  git clone git@github.com:scylladb/python-driver.git scylla-python-driver &
  git clone git@github.com:scylladb/scylla.git scylla &
  git clone git@github.com:scylladb/scylla-ccm.git scylla-ccm &
  git git@github.com:scylladb/python-driver-matrix.git python-driver-matrix &
  wait

Running locally

• Execute the main.py wrapper like:

  • Running with scylla-python-driver:

  • Scylla driver:

    # Run all standard tests on latest python-driver tag (--versions 1)
    python3 main.py ../scylla-python-driver ../scylla --tests tests.integration.standard --driver-type scylla --versions 1 --protocols 3,4
    
    # Run all standard tests with specific python-driver tag (--versions 3.25.0-scylla)
    python3 main.py ../scylla-python-driver ../scylla --tests tests.integration.standard --driver-type scylla --versions 3.25.0-scylla --protocols 3,4

  • Datastax driver:

    # Run all standard tests on latest python-driver tag (--versions 1)
    python3 main.py ../datastax-python-driver ../scylla --tests tests.integration.standard --driver-type datastax --versions 1 --protocols 3,4
    
    # Run all standard tests with specific python-driver tag (--versions 3.25.0)
    python3 main.py ../datastax-python-driver ../scylla --tests tests.integration.standard --driver-type datastax --versions 3.25.0-scylla --protocols 3,4

  • Running with docker image:

  • Scylla driver:

    export INSTALL_DIRECTORY=../scylla
    
    # running with anything other then release, this should be added
    # export CASSANDRA_DIR=../scylla/build/debug
    ./scripts/run_test.sh python main.py ../scylla-python-driver $INSTALL_DIRECTORY --tests tests.integration.standard --driver-type scylla --versions 3.25.0 --protocol 3,4

  • Datastax driver:

    export INSTALL_DIRECTORY=../scylla
    
    # running with anything other then release, this should be added
    # export CASSANDRA_DIR=../scylla/build/debug
    ./scripts/run_test.sh python main.py ../datastax-python-driver $INSTALL_DIRECTORY --tests tests.integration.standard --driver-type datastax --versions 3.25.0 --protocol 3,4

  • Running with relocatable packages:
  • Scylla driver:

    export SCYLLA_VERSION=unstable/master:2021-12-16T09:10:53Z
    ./scripts/run_test.sh python main.py ../scylla-python-driver --tests tests.integration.standard --driver-type scylla --versions 3.25.0 --protocol 3,4 --scylla-version $SCYLLA_VERSION

  • Datastax driver:

    export SCYLLA_VERSION=unstable/master:2021-12-16T09:10:53Z
    ./scripts/run_test.sh python main.py ../datastax-python-driver --tests tests.integration.standard --driver-type datastax --versions 3.25.0 --protocol 3,4 --scylla-version $SCYLLA_VERSION

    export SCYLLA_VERSION=unstable/master:265 ./scripts/run_test.sh python main.py ../python-driver --tests tests.integration.standard --versions 3.9.0 --protocol 3 --scylla-version $SCYLLA_VERSION

  • Running from PyCharm:
  • Create a basic Python configure.
  • Working directory value is: /home/oren/Desktop/github/python-driver-matrix
  • Script path value is: main.py
  • Parameters value are:

    /home/oren/Desktop/github/python-driver /home/oren/Desktop/github/scylla
    --driver-type scylla
    --scylla-version unstable/master/2022-01-03T13_22_36Z
    --versions 1

  • Environment variables are:

    PYTHONUNBUFFERED=1;
    MAPPED_SCYLLA_VERSION=3.11.4

Uploading docker images

When doing changes to requirements.txt, or any other change to docker image, it can be uploaded like this:

    export MATRIX_DOCKER_IMAGE=scylladb/scylla-python-driver-matrix:python3.9-$(date +'%Y%m%d')
    docker build ./scripts -t ${MATRIX_DOCKER_IMAGE}
    docker push ${MATRIX_DOCKER_IMAGE}
    echo "${MATRIX_DOCKER_IMAGE}" > scripts/image

Note: you'll need permissions on the scylladb dockerhub organization for uploading images