Documentation on how to run Airbyte behind Traefik

[codefriar]

Tell us about the documentation you'd like us to add or update

I've seen the CEO's comments on HN about adjusting the APIURL, but i'm not able to get it to function correctly (throws a 405 on api/v1/workspaces/get...

I've established a traefik network, and the airbyte_network and pulling from the official docker-compose file put this together

```yml

version: '3'

This networks section required for the package to be visible to Traefik.

DO NOT Remove

networks: traefik_network: external: name: vivumlab_traefik airbyte_network:

services: init: image: airbyte/init:0.18.2-alpha container_name: init command: /bin/sh -c "./scripts/create_mount_directories.sh /local_parent /tmp /tmp/airbyte_local" environment:

• LOCAL_ROOT=/tmp/airbyte_local
• HACK_LOCAL_ROOT_PARENT=/tmp volumes:
• /tmp:/local_parent networks:

  - traefik_network

• airbyte_network db: image: airbyte/db:0.18.2-alpha container_name: airbyte-db environment:
• POSTGRES_USER=docker
• POSTGRES_PASSWORD=docker
• POSTGRES_DB=db-airbyte volumes:
• db:/var/lib/postgresql/data networks:
• airbyte_network seed: image: airbyte/seed:0.18.2-alpha container_name: airbyte-data-seed

  Pre-populate the volume if it is empty.

  See: https://docs.docker.com/storage/volumes/#populate-a-volume-using-a-container

  volumes:

• data:/app/seed networks:
• airbyte_network scheduler: image: airbyte/scheduler:0.18.2-alpha container_name: airbyte-scheduler environment:
• DATABASE_USER=docker
• DATABASE_PASSWORD=docker
• DATABASE_URL=jdbc:postgresql://db:5432/airbyte
• WAIT_BEFORE_HOSTS=5
• WAIT_HOSTS_TIMEOUT=45
• WAIT_HOSTS=db:5432
• WORKSPACE_ROOT=/tmp/workspace
• WORKSPACE_DOCKER_MOUNT=airbyte_workspace
• LOCAL_DOCKER_MOUNT=/tmp/airbyte_local
• CONFIG_ROOT=/data
• TRACKING_STRATEGY=segment
• AIRBYTE_VERSION=0.18.2-alpha
• AIRBYTE_ROLE=${AIRBYTE_ROLE:-} volumes:
• /var/run/docker.sock:/var/run/docker.sock
• workspace:/tmp/workspace
• /tmp/airbyte_local:/tmp/airbyte_local
• data:/data networks:
• traefik_network
• airbyte_network server: image: airbyte/server:0.18.2-alpha container_name: airbyte-server environment:
• DATABASE_USER=docker
• DATABASE_PASSWORD=docker
• DATABASE_URL=jdbc:postgresql://db:5432/airbyte
• WAIT_BEFORE_HOSTS=5
• WAIT_HOSTS_TIMEOUT=45
• WAIT_HOSTS=db:5432
• CONFIG_ROOT=/data
• TRACKING_STRATEGY=segment
• AIRBYTE_VERSION=0.18.2-alpha
• AIRBYTE_ROLE=${AIRBYTE_ROLE:-}
• WORKSPACE_ROOT=/tmp/workspace ports:
• 8001:8001 volumes:
• workspace:/tmp/workspace
• data:/data networks:

  - traefik_network

• airbyte_network webapp: image: airbyte/webapp:0.18.2-alpha container_name: airbyte-webapp environment:
• AIRBYTE_ROLE=${AIRBYTE_ROLE:-}
• AIRBYTE_VERSION=0.18.2-alpha
• API_URL=/api/v1/
• IS_DEMO=${IS_DEMO:-}
• PAPERCUPS_STORYTIME=${PAPERCUPS_STORYTIME:-}
• TRACKING_STRATEGY=segment networks:
• traefik_network
• airbyte_network labels:
• "traefik.http.services.airbyte.loadbalancer.server.scheme=http"
• "traefik.http.services.airbyte.loadbalancer.server.port=80" {% include './labels.j2' %} airbyte-temporal: image: temporalio/auto-setup:1.7.0 container_name: airbyte-temporal ports:
• 7233:7233 environment:
• DB=postgresql
• DB_PORT=5432
• POSTGRES_USER=docker
• POSTGRES_PWD=docker
• POSTGRES_SEEDS=db
• DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development.yaml volumes:
• ./temporal/dynamicconfig:/etc/temporal/config/dynamicconfig networks:

  - traefik_network

• airbyte_network volumes: workspace: name: airbyte_workspace data: name: airbyte_data db: name: airbyte_db

I'm trying to establish this as a service for vivumlab (https://github.com/VivumLab/VivumLab) so it's rapidly and easily deployable for home-lab use. Any advice is appreciated.

┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1200367912513076/1200368099789921) by [Unito](https://www.unito.io)

[michel-tricot]

@codefriar Would love to help on fixing this, it will likely be easier on a zoom. Do you have time next week to discuss it? I will contact you on slack to set this up.

[codefriar]

For anyone else wondering, here's the magic for traefik 2.

The web front end will need standard traefik labels, but the api will need tags that add && PathPrefix(/api) and therefor look like this:

"traefik.http.routers.{{ service_item }}-api-http.rule=Host({{ service_domain }}) && PathPrefix(/api)"

[hongbo-miao]

Thanks @codefriar !

For future people who come here, I want to point out today is actually just simply switching Terraform provider to server_url = "${var.airbyte_url}/api/public/v1". Here airbyte_url points to airbyte-webapp-svc.airbyte-namespace.

It helps

• Terraform works well with new endpoint.
• This new endpoint is latest. airbyte-api-server is actually out-of-date today. The service underneath this new endpoint fixed bug like this one.
• If you use Helm chart to deploy, you can disable airbyte-api-server which is one less server!

  airbyte-api-server:
  enabled: false

• Ingress only needs to expose one endpoint now.