Eclipse Otterdog

Introduction

Otterdog is a tool to manage GitHub organizations at scale using a configuration as code approach. It is actively developed by the Eclipse Foundation and used to manage its numerous projects hosted on GitHub.

Quickstart

To install and use the cli part of otterdog you have to install the following:

git (mandatory): install using apt install git
otterdog (mandatory): install using pipx install otterdog
bitwarden cli tool (optional): install using snap install bw
pass cli tool (optional): install using apt install pass

Otterdog Presentation @ Open Source Summit 2023

Default Configuration used @ Eclipse Foundation

Documentation

The documentation is available at otterdog.readthedocs.io.

Build instructions

System requirements:

python3.11+ (mandatory): e.g. install using apt install python3 or use pyenv install 3.12
git (mandatory): install using apt install git
poetry (mandatory): install using pipx install poetry
dynamic versioning plugin (mandatory): install using pipx inject poetry "poetry-dynamic-versioning[plugin]"
bitwarden cli tool (optional): install using snap install bw
pass cli tool (optional): install using apt install pass

Building Steps

Create a virtual python environment and install necessary python dependencies using poetry:

$ make init

Running make init will also install poetry and the dynamic versioning plugin if it is not installed yet.

Testing build

$ ./otterdog.sh -h

Setup

The general configuration for supported organizations and their corresponding credentials in order to access their GitHub settings has to be placed in a json file (default: otterdog.json, can be changed with the -c flag), e.g.:

{
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "<bitwarden | pass>",
        "item_id" : "39adacc9-2b51-41a9-a27e-ac7c00eea6a5"
      }
    }
  ]
}

Credentials

Otterdog needs certain credentials to access information from an organization and its repositories on GitHub:

username / password / 2FA seed
API token

The login / username / 2FA seed are required to access the web interface of GitHub in order to retrieve certain settings that are not accessible via its rest / graphql API.

The GitHub api token needs to have the following scopes enabled:

repo
workflow
admin:org
admin:org_hook
delete_repo

The credentials can be stored in different providers (bitwarden, pass).

Bitwarden

When using bitwarden to store the credentials, you need to enter a valid item id as additional credential data:

{
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "bitwarden",
        "item_id" : "<bitwarden item id>"
      }
    }
  ]
}

The item stored in bitwarden needs to contain the following information (a sample json output of such an item):

{
  "object": "item",
  "id": "<bitwarden item id>",
  "name": "<item name>",
  "fields": [
    {
      "name": "api_token_admin",
      "value": "<github API token>"
    }
  ],
  "login": {
    "username": "<github username>",
    "password": "<github password>",
    "totp": "<github TOTP text code>"
  }
}

Mandatory items:

Field with name "api_token_admin" and as value the GitHub token to access the organization
login.username of a user that can access the organization with enabled 2FA
login.password the password of that user
login.totp the TOTP text code

Pass

When using pass to store the credentials, you need to enter fully qualified pass names to access the various required credential data:

{
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "pass",
        "api_token": "<path/to/api_token>",
        "username": "<path/to/username>",
        "password": "<path/to/password>",
        "2fa_seed": "<path/to/2fa_seed>"
      }
    }
  ]
}

In case your password storage dir is not located at the default location, you can configurate that in the defaults:

{
  "defaults": {
    "pass": {
      "password_store_dir": "path/to/storage/dir"
    }
  }
}

Usage

Run the import operation to retrieve the current live configuration for an organization:

$ otterdog.sh import <organization>

The created configuration file for the organization can be found at <data-directory>/orgs/<organization>.jsonnet

Run the plan operation to highlight differences between the live configuration and the written configuration:

$ otterdog.sh plan <organization>

Run apply operation to reflect the written configuration on github itself:

$ otterdog.sh apply <organization>

eclipse-csi / otterdog

readme