TDP Lib

tdp-lib is a Python library built on top of Ansible. It is designed for managing TDP clusters, offering advanced tools to extend Ansible's capabilities. Key features include:

• Creating a Directed Acyclic Graph (DAG) of tasks to manage dependencies and relationships between services and components.
• Centralized configuration management through a unified variable definition system.

tdp-lib can be utilized in various ways: as a Python library, through an admin CLI, via a REST API (see tdp-server), or through a web interface (see tdp-ui).

Pre-requisites

To use tdp-lib, ensure you have the following prerequisites:

• Python 3.9 or higher. Higher version are not guarantee to work, only 3.9 is tested.
• A relational database management system (RDBMS), such as PostgreSQL or SQLite.

Optional dependencies for DAG visualization:

• Graphviz for graphical representation of DAGs.

Installation

Set the following environment variables:

• TDP_COLLECTION_PATH: Specifies the file path(s) to the necessary collection(s). tdp-collection is mandatory. Multiple collections can be specified, separated by a colon : (e.g., tdp-collection-extras, tdp-observability).
• TDP_RUN_DIRECTORY: Path to the working directory of the TDP deployment (where ansible.cfg, inventory.ini, and topology.ini are located).
• TDP_DATABASE_DSN: Database DSN (Data Source Name) for the chosen RDBMS.
• TDP_VARS: Path to the folder containing configuration variables.

Ensure Ansible is configured to use the tosit.tdp.inventory plugin. Example ansible.cfg:

[defaults]
inventory=your_inventory,..,~/tdp_vars

[inventory]
enable_plugins = tosit.tdp.inventory,..,your_plugins

Install the library:

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate
# Install the dependencies
pip install "tdp-lib[visualization]@https://github.com/TOSIT-IO/tdp-lib/tarball/master"
# Initialize the database and tdp_vars
tdp init

CLI Usage

[!NOTE] This section is a work in progress.

tdp --help

Contributing

Contributions are welcome! Here are some guidelines specific to this project:

• Use Poetry for development:

  # Install Poetry
  curl -sSL https://install.python-poetry.org | python3 -
  # Install the dependencies
  poetry install -E postgresql-binary -E mysql

• Commit messages must adhere to the Conventional Commits standard.

• Run tests on all supported databases before submitting a PR:

  docker compose -f dev/docker-compose.yaml up -d
  poetry run pytest tests --database-dsn 'postgresql+psycopg2://postgres:postgres@localhost:5432/tdp' --database-dsn 'mysql+pymysql://mysql:mysql@localhost:3306/tdp' --database-dsn 'mysql+pymysql://mariadb:mariadb@localhost:3307/tdp'
  docker compose -f dev/docker-compose.yaml down -v

• Format and lint code using (Black) and (Ruff):

  poetry run task precommit-fix

Developer Documentation

Developer documentation is available here: docs/Developer

Docstrings are used to generate Sphinx documentation. Install the `docs`` extra dependency:

poetry install -E docs

Build the documentation:

poetry run task docs

The built documentation is available at docs/_build/html/index.html.