search

jittering / traefik-kop

A dynamic docker->redis->traefik discovery agent
MIT License
179 stars 13 forks source link

Existing Redis Keys Removed Upon Adding New Server's Redis Keys #28

Closed lued closed 10 months ago

lued commented 10 months ago

Hello,

Thanks for making this project. I am experiencing a strange behaviour of a relatively trivial implementation. I have 3 servers:

1. proxy: Traefik & Redis

docker-compose.yml ``` networks: wan: name: wan driver: bridge ipam: driver: default config: - subnet: 10.0.100.0/24 docker_proxy: name: docker_proxy external: false traefik-kop: name: traefik-kop external: false volumes: traefik-acme: services: traefik: image: traefik hostname: traefik container_name: traefik restart: unless-stopped command: - --global.sendAnonymousUsage=false - --entryPoints.web.address=:80 - --entryPoints.websecure.address=:443 - --entrypoints.web.http.redirections.entrypoint.to=websecure - --entrypoints.web.http.redirections.entrypoint.scheme=https # Allow these IPs to set the X-Forwarded-* headers - Cloudflare IPs: https://www.cloudflare.com/ips/ - --entrypoints.websecure.forwardedHeaders.trustedIPs=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22 - --api - --log - --metrics.influxdb2=true - --metrics.influxdb2.address=http://influx:8086 - --metrics.influxdb2.token=${INFLUX_TOKEN} - --metrics.influxdb2.org=${INFLUX_ORG} - --metrics.influxdb2.bucket=${INFLUX_BUCKET} - --metrics.influxdb2.addEntryPointsLabels=true - --metrics.influxdb2.addrouterslabels=true - --metrics.influxdb2.addServicesLabels=true - --metrics.influxdb2.additionallabels.host=proxy - --metrics.influxdb2.additionallabels.environment=local - --log.level=DEBUG # DEBUG, INFO, WARN, ERROR, FATAL, PANIC - --providers.providersThrottleDuration=2s - --providers.redis - --providers.redis.endpoints=traefik-redis:6379 - --providers.redis.password=${REDIS_PASS} - --providers.docker - --providers.docker.watch=true - --providers.docker.endpoint=tcp://docker_proxy:2375 #unix:///var/run/docker.sock - --providers.docker.defaultrule=Host(`{{ index .Labels "com.docker.compose.service" }}`) - --providers.docker.exposedByDefault=false - --providers.docker.network=wan - --providers.docker.swarmMode=false - --providers.file.directory=/etc/traefik/dynamic-conf/ - --providers.file.watch=true - --certificatesResolvers.dns-cloudflare.acme.caServer=https://acme-v02.api.letsencrypt.org/directory #- --certificatesResolvers.dns-cloudflare.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory - --certificatesResolvers.dns-cloudflare.acme.email=$CLOUDFLARE_EMAIL - --certificatesResolvers.dns-cloudflare.acme.storage=/acme/acme.json - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.provider=cloudflare - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.resolvers=1.1.1.1:53,1.0.0.1:53 - --certificatesResolvers.dns-cloudflare.acme.dnsChallenge.delayBeforeCheck=90 networks: - wan - docker_proxy - traefik-kop security_opt: - no-new-privileges:true ports: - 80:80 - 443:443 volumes: - traefik-acme:/acme - ${DOCKERDIR}/traefik/dynamic/:/etc/traefik/dynamic-conf/ environment: - CF_API_EMAIL=${CLOUDFLARE_EMAIL} - CF_API_KEY=${CLOUDFLARE_API_KEY} - TZ=${TZ} labels: - traefik.enable=true # Middleware Rules # # Basic Authentication - traefik.http.middlewares.basic-auth.basicAuth.realm=Traefik Basic Authentication - traefik.http.middlewares.basic-auth.basicAuth.users=${BASIC_AUTH_USER}:${BASIC_AUTH_PASS} # # Rate Limit - traefik.http.middlewares.rate-limit.rateLimit.average=100 - traefik.http.middlewares.rate-limit.rateLimit.burst=50 # # Secure Headers - traefik.http.middlewares.security-headers.headers.accesscontrolallowmethods=GET, OPTIONS, PUT - traefik.http.middlewares.security-headers.headers.accesscontrolmaxage=100 - traefik.http.middlewares.security-headers.headers.addvaryheader=true - traefik.http.middlewares.security-headers.headers.hostsproxyheaders=X-Forwarded-Host - traefik.http.middlewares.security-headers.headers.sslredirect=true - traefik.http.middlewares.security-headers.headers.sslproxyheaders.X-Forwarded-Proto=https - traefik.http.middlewares.security-headers.headers.stsseconds=63072000 - traefik.http.middlewares.security-headers.headers.stsincludesubdomains=true - traefik.http.middlewares.security-headers.headers.stspreload=true - traefik.http.middlewares.security-headers.headers.forcestsheader=true - traefik.http.middlewares.security-headers.headers.framedeny=true - traefik.http.middlewares.security-headers.headers.contenttypenosniff=true - traefik.http.middlewares.security-headers.headers.browserxssfilter=true - traefik.http.middlewares.security-headers.headers.referrerpolicy=same-origin # Middleware Chains - traefik.http.middlewares.no-auth.chain.middlewares=rate-limit,security-headers - traefik.http.middlewares.basic-auth.chain.middlewares=rate-limit,security-headers,basic-auth ## Middlewares - traefik.http.routers.traefik-rtr.middlewares=no-auth@docker # Services - API - traefik.http.routers.traefik-rtr.service=api@internal # HTTP Routers - traefik.http.routers.traefik-rtr.entrypoints=websecure - traefik.http.routers.traefik-rtr.rule=Host(`traefik.$MAIN_DOMAIN`) - traefik.http.routers.traefik-rtr.tls=true # Setting TLS to $DOMAIN# - traefik.http.routers.traefik-rtr.tls.certResolver=dns-cloudflare - traefik.http.routers.traefik-rtr.tls.domains[0].main=$MAIN_DOMAIN - traefik.http.routers.traefik-rtr.tls.domains[0].sans=*.$MAIN_DOMAIN traefik-redis: hostname: traefik-redis container_name: traefik-redis image: redis:alpine restart: always command: redis-server --requirepass ${REDIS_PASS} networks: - traefik-kop healthcheck: test: ["CMD", "redis-cli", "-p", "6379", "ping"] interval: 30s timeout: 10s retries: 3 ports: - 6379:6379 volumes: - ${DOCKERDIR}/traefik/redis/config:/usr/local/etc/redis - ${DOCKERDIR}/traefik/redis/data:/data ``` env file ``` DOCKERDIR=/srv/docker ... MAIN_DOMAIN=lued.tech ... # traefik-kop REDIS_ADDR=[REDIS FQDN]:6379 REDIS_PASS=[REDIS PASSWORD] ``` relevant traefik container log: ![image](https://github.com/jittering/traefik-kop/assets/10970148/02015764-f2c4-4f1f-b3da-59db2f9e8b33) Only showing this log as there is nothing relevant in Traefik besides it working as expected and finding the dynamic rule provided by redis upon traefik-kop reading the container labels on other servers.

2. security: traefik-kop & VaultWarden (A compose container)

docker-compose.yml ``` networks: docker_proxy: name: docker_proxy external: false services: traefik-kop: image: ghcr.io/jittering/traefik-kop:latest hostname: traefik-kop container_name: traefik-kop restart: always networks: - docker_proxy environment: - REDIS_ADDR=${REDIS_ADDR} - REDIS_PASS=${REDIS_PASS} - BIND_IP=${HOST_IP} - DOCKER_HOST=tcp://docker_proxy:2375 - POLL_INTERVAL=10 - DEBUG=1 ``` env file ``` # traefik-kop REDIS_ADDR=[REDIS FQDN]:6379 REDIS_PASS=[REDIS PASSWORD] HOST_IP=[HOST IP] ``` docker-compose.yml for vaultwarden ``` networks: wan: name: wan driver: bridge ipam: driver: default config: - subnet: 10.0.100.0/24 services: vaultwarden: image: vaultwarden/server container_name: vaultwarden hostname: vault restart: always networks: - wan ports: - 2122:80 - 3012:3012 environment: - ADMIN_TOKEN=${ADMIN_TOKEN} - WEBSOCKET_ENABLED=true - PASSWORD_ITERATIONS=600000 - SIGNUPS_ALLOWED=false - LOG_FILE=/data/vaultwarden.log - LOG_LEVEL=warn - EXTENDED_LOGGING=true - TZ=$TZ - SMTP_HOST=${SMTP_SERVER} - SMTP_FROM=vault@${MAIN_DOMAIN} - SMTP_FROM_NAME=${FROM_NAME} - SMTP_SECURITY=${SMTP_SECURITY} - SMTP_PORT=${SMTP_PORT} - SMTP_USERNAME=${SMTP_USER} - SMTP_PASSWORD=${SMTP_PASS} - SMTP_TIMEOUT=15 - DOMAIN=https://vault.${MAIN_DOMAIN} volumes: - ${DOCKERDIR}/vaultwarden:/data labels: - "traefik.enable=true" # routers & services - "traefik.http.routers.vaultwarden.tls=true" - "traefik.http.routers.vaultwarden.entrypoints=websecure" - "traefik.http.routers.vaultwarden.tls.certresolver=dns-cloudflare" - "traefik.http.routers.vaultwarden.rule=Host(`vault.${MAIN_DOMAIN}`)" - "traefik.http.routers.vaultwarden.service=vaultwarden" - "traefik.http.services.vaultwarden.loadbalancer.server.scheme=http" - "traefik.http.services.vaultwarden.loadbalancer.server.port=2122" # websocket - "traefik.http.routers.vaultwarden-ws.tls=true" - "traefik.http.routers.vaultwarden-ws.entrypoints=websecure" - "traefik.http.routers.vaultwarden-ws.tls.certresolver=dns-cloudflare" - "traefik.http.routers.vaultwarden-ws.rule=Host(`vault.${MAIN_DOMAIN}`) && Path(`/notifications/hub`)" - "traefik.http.routers.vaultwarden-ws.service=vaultwarden-ws" - "traefik.http.services.vaultwarden-ws.loadbalancer.server.scheme=http" - "traefik.http.services.vaultwarden-ws.loadbalancer.server.port=3012" ``` env file for vaultwarden ``` DOCKERDIR=/srv/docker TZ=[my timezone] MAIN_DOMAIN=[my domain] ... ```

3. business: traefik-kop & freescout (Another compose container)

traefik-kop docker-compose.yml ``` networks: docker_proxy: name: docker_proxy external: false services: traefik-kop: image: ghcr.io/jittering/traefik-kop:latest hostname: traefik-kop container_name: traefik-kop restart: always networks: - docker_proxy environment: - REDIS_ADDR=${REDIS_ADDR} - REDIS_PASS=${REDIS_PASS} - BIND_IP=${HOST_IP} - DOCKER_HOST=tcp://docker_proxy:2375 - POLL_INTERVAL=10 - DEBUG=1 ``` env file for traefik-kop ``` # traefik-kop REDIS_ADDR=[REDIS FQDN]:6379 REDIS_PASS=[REDIS PASSWORD] HOST_IP=[HOST IP] ``` docker-compose.yml for freescout ``` networks: wan: name: wan driver: bridge ipam: driver: default config: - subnet: 10.0.100.0/24 services: freescout-app: image: tiredofit/freescout hostname: freescout-app container_name: freescout-app links: - freescout-db volumes: ... environment: ... networks: - wan ports: - 2140:80 restart: unless-stopped labels: - "traefik.enable=true" - "traefik.http.routers.freescout.tls=true" - "traefik.http.routers.freescout.entrypoints=websecure" - "traefik.http.routers.freescout.tls.certresolver=dns-cloudflare" - "traefik.http.routers.freescout.rule=Host(`it.${MAIN_DOMAIN}`)" - "traefik.http.routers.freescout.service=freescout" - "traefik.http.services.freescout.loadbalancer.server.scheme=http" - "traefik.http.services.freescout.loadbalancer.server.port=2140" freescout-db: image: tiredofit/mariadb container_name: freescout-db volumes: ... environment: ... networks: - wan restart: unless-stopped ``` env file for freescout ``` DOCKERDIR=/srv/docker TZ=[tz] MAIN_DOMAIN=[my domain] ... ```

Behaviour:

image

It appears redis is purging upon key ingestion. Am I missing something here?

Thanks

lued commented 10 months ago

Found the problem. Each server has identical container names for traefik-kop. Upon checking redis-cli > MONITOR, it became obvious: simply change the container names.....

1698483733.859007 [0 10.0.1.22:44596] "smembers" "traefik_http_middlewares@traefik-kop"
1698483733.859308 [0 10.0.1.22:44596] "sadd" "traefik_http_routers@traefik-kop_new" "vaultwarden@docker" "vaultwarden-ws@docker"
1698483733.859531 [0 10.0.1.22:44596] "sdiff" "traefik_http_routers@traefik-kop" "traefik_http_routers@traefik-kop_new"
1698483733.860524 [0 10.0.1.22:44596] "keys" "traefik/http/services/freescout/*"
1698483733.860687 [0 10.0.1.22:44596] "unlink" "traefik/http/services/freescout/loadBalancer/servers/0/url" "traefik/http/services/freescout/loadBalancer/passHostHeader"
chetan commented 10 months ago

@lued glad you were able to figure it out. It's actually the hostname that's used for the identifier in redis there. I assume you changed both when you fixed it?

hostname: traefik
lued commented 10 months ago

@chetan, I didn't have to change the server running traefik, all I had changed were the hostnames of the containers on servers running traefik-kop.

I had originally change the entire container name as well, but with the information that I just need to change the "hostname:" information, I ended up with this for each new server's template

  traefik-kop:
    image: ghcr.io/jittering/traefik-kop:latest
    hostname: traefik-kop-${SERVER_HOSTNAME}
    ...

where the .env file has SERVER_HOSTNAME equal to the hostname.

Thanks for the great work on this project.