Cannot connect to server error

tehniemer commented 2 weeks ago

I'm unable to connect this to my server, every attempt ends with a Could not connect to server error. I've verified my URL and API are correct, even generated a new API key, nothing helps.

Not sure what my next steps should be.

Protonova commented 2 weeks ago

Just my two cents here... I had a similar problem. Are you running Traefik? Have you tried removing https://? For example on my installation I have tubearchivist.***.***.com:7081 7081 being the port I have the load balancer running on (can be anything on your end).

tehniemer commented 2 weeks ago

I am running Traefik and tried your suggestion with no change in behavior.

Protonova commented 2 weeks ago

Then there's something wrong with your traefik config. I'm not sure if this is the right place for this issue to be opened. I'm just randomly contributing since I had the same issue a few hours ago when I put TA behind the load balancer.

If the admins don't mind, sure I'll help troubleshoot the issue. You need to share the labels you're using for traefik in your docker. As well as the configs you have for your traefik instance. Please share them using code blocks and remember to remove sensitive information :)

tehniemer commented 2 weeks ago

Thanks, I appreciate it. I've tried with and without Authelia, which has a global bypass rule for '^/api([/?].*)?$'

Here's my relevant compose stack:

version: "3.8"
services:
  traefik:
    image: traefik:latest
    container_name: traefik
    restart: always
    #    dns:
    #      - 1.1.1.1
    #      - 1.0.0.1
    networks:
      - traefik
      - socket_proxy
      - internal
    ports:
      - $TRAEFIKHTTP_PORT:80
      - $TRAEFIKHTTPS_PORT:443
    security_opt:
      - no-new-privileges=true
    logging:
      options:
        max-size: 2m
        max-file: "3"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/traefik2/traefik.yml:/etc/traefik/traefik.yml:ro # config location - touch file before starting container
      - /opt/docker/traefik2/rules:/rules:ro # file provider directory
      - /opt/docker/traefik2/acme/acme.json:/acme.json # cert location - touch file and change permissions to 600 before starting container
      - /opt/docker/traefik2/traefik.log:/traefik.log # for fail2ban - touch file before starting container
      - /opt/docker/traefik2/access.log:/access.log # touch file before starting container
    environment:
      TZ: $TIMEZONE
      #      CF_API_EMAIL: $CLOUDFLARE_EMAIL
      #      CF_API_KEY: $CLOUDFLARE_API_KEY
      CF_DNS_API_TOKEN: $CLOUDFLARE_DNS_API_TOKEN
      DOMAINNAME: $DOMAINNAME
    labels:
      - homepage.group=Containers
      - homepage.name=Traefik
      - homepage.icon=traefik
      - homepage.description=Reverse Proxy
      - homepage.href=https://traefik.$DOMAINNAME
      - traefik.enable=true
      - traefik.http.routers.traefik-rtr.entrypoints=https
      - traefik.http.routers.traefik-rtr.rule=Host(`traefik.$DOMAINNAME`)
      - traefik.http.routers.traefik-rtr.service=api@internal
      - traefik.http.routers.traefik-rtr.middlewares=chain-authelia@file #Authelia Authentication
  tubearchivist:
    image: bbilly1/tubearchivist
    container_name: tubearchivist
    restart: unless-stopped
    networks:
      - traefik
      - internal
    ports:
      - $TUBEARCHIVIST_PORT:8000
    security_opt:
      - no-new-privileges=true
    logging:
      options:
        max-size: 2m
        max-file: "3"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/tubearchivist:/cache
      - $MEDIA_DIR/.temp/TubeArchivist/download:/cache/download
      - $MEDIA_DIR/.temp/TubeArchivist/import:/cache/import
      - $MEDIA_DIR/TubeArchivist:/youtube
    environment:
      TZ: $TIMEZONE
      HOST_UID: $PUID
      HOST_GID: $PGID
      TA_HOST: $HOST $HOST:$TUBEARCHIVIST_PORT https://tubearchivist.$DOMAINNAME
      TA_USERNAME: $TUBEARCHIVIST_USER
      TA_PASSWORD: $TUBEARCHIVIST_PASS
#      TA_ENABLE_AUTH_PROXY: true
#      TA_AUTH_PROXY_USERNAME_HEADER: HTTP_REMOTE_USER
#      TA_AUTH_PROXY_LOGOUT_URL: https://authelia.$DOMAINNAME
      ES_URL: http://tubearchivist-es:9200
      ELASTIC_PASSWORD: $ELASTIC_PASS
      REDIS_HOST: tubearchivist-redis
    healthcheck:
      test:
        - CMD
        - curl
        - -f
        - http://localhost:8000/health
      interval: 2m
      timeout: 10s
      retries: 3
      start_period: 30s
    labels:
      - homepage.group=Containers
      - homepage.name=TubeArchivist
      - homepage.icon=tubearchivist
      - homepage.description=YouTube Downloader
      - homepage.href=https://tubearchivist.$DOMAINNAME
      - homepage.siteMonitor=http://$HOST:$TUBEARCHIVIST_PORT
      - traefik.enable=true
      - traefik.http.routers.tubearchivist-rtr.entrypoints=https
      - traefik.http.routers.tubearchivist-rtr.rule=Host(`tubearchivist.$DOMAINNAME`)
      - traefik.http.routers.tubearchivist-rtr.middlewares=chain-authelia@file #Authelia Authentication
      #- traefik.http.routers.tubearchivist-rtr.middlewares=chain-no-auth@file #No Authentication
      - traefik.http.routers.tubearchivist-rtr.service=tubearchivist-svc
      - traefik.http.services.tubearchivist-svc.loadbalancer.server.port=8000
  tubearchivist-es:
    image: elasticsearch:8.14.3
    container_name: tubearchivist-es
    restart: unless-stopped
    networks:
      - traefik
      - internal
    ulimits:
      memlock:
        soft: -1
        hard: -1
    security_opt:
      - no-new-privileges=true
    logging:
      options:
        max-size: 2m
        max-file: 3
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/tubearchivist-es:/usr/share/elasticsearch/data
    environment:
      - ELASTIC_PASSWORD=$ELASTIC_PASS
      - ES_JAVA_OPTS=-Xms1g -Xmx1g
      - xpack.security.enabled=true
      - discovery.type=single-node
      - path.repo=/usr/share/elasticsearch/data/snapshot
    labels:
      - homepage.group=Containers
      - homepage.name=Tubearchivist Elasticpass
      - homepage.icon=elasticpass
  tubearchivist-redis:
    image: redis/redis-stack-server:latest
    container_name: tubearchivist-redis
    restart: always
    networks:
      - internal
    security_opt:
      - no-new-privileges=true
    logging:
      options:
        max-size: 2m
        max-file: "3"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/tubearchivist-redis:/data
    labels:
      - homepage.group=Containers
      - homepage.name=Tubearchivist Redis
      - homepage.icon=redis
networks:
  traefik:
    external: true
  internal:
    external: true
  socket_proxy:
    external: true

Traefik config:

global:
  checkNewVersion: true
  sendAnonymousUsage: false

api:
  dashboard: true
  insecure: true

log:
  level: 'INFO' # (Default: error) DEBUG, INFO, WARN, ERROR, FATAL, PANIC
  filePath: '/traefik.log'

accessLog:
  filePath: '/access.log'
  bufferingSize: 100 # Configuring a buffer of 100 lines
  filters:
    statusCodes:
      - '204-299'
      - '400-499'
      - '500-599'

ping: {}

entryPoints:
  # Not used in apps, but redirect everything from HTTP to HTTPS
  http:
    address: :80
    forwardedHeaders:
      trustedIPs: &trustedIps
        # Local IP range
        #- 127.0.0.1/32
        #- 10.0.0.0/8
        #- 172.16.0.0/12
        #- 192.168.0.0/16
        #- fc00::/7
        # Start of Clouflare public IP list for HTTP requests, remove this if you don't use it
        - 173.245.48.0/20
        - 103.21.244.0/22
        - 103.22.200.0/22
        - 103.31.4.0/22
        - 141.101.64.0/18
        - 108.162.192.0/18
        - 190.93.240.0/20
        - 188.114.96.0/20
        - 197.234.240.0/22
        - 198.41.128.0/17
        - 162.158.0.0/15
        - 104.16.0.0/12
        - 172.64.0.0/13
        - 131.0.72.0/22
        - 2400:cb00::/32
        - 2606:4700::/32
        - 2803:f800::/32
        - 2405:b500::/32
        - 2405:8100::/32
        - 2a06:98c0::/29
        - 2c0f:f248::/32
        # End of Cloudlare public IP list
    http:
      redirections:
        entryPoint:
          to: https
          scheme: https

  # HTTPS endpoint, with domain wildcard
  https:
    address: :443
    forwardedHeaders:
      # Reuse list of Cloudflare Trusted IP's above for HTTPS requests
      trustedIPs: *trustedIps
    http:
      tls:
        certresolver: 'dns-cloudflare'
        domains:
          - main: 'mydomain.com'
          - sans:
            - '*.mydomain.com'
#        options: 'tls-opts@file'

providers:
  providersThrottleDuration: 2s
  # Docker provider for connecting all apps that are inside of the docker network
  docker:
    endpoint: 'tcp://socket-proxy:2375' #'unix:///var/run/docker.sock'
    network: 'traefik'
    exposedByDefault: false
    watch: true
  # File provider for connecting things that are outside of docker / defining middleware
  file:
    directory: '/rules'
    watch: true
#  redis:
#    endpoints:
#      - 'redis:6379'
#    db: 2

certificatesResolvers:
  dns-cloudflare:
    acme:
      email: 'jon.doe@gmail.com'
      storage: '/acme.json'
      dnsChallenge:
        provider: 'cloudflare'
        delayBeforeCheck: '5'
        resolvers:
          - '1.1.1.1:53'
          - '1.0.0.1:53'

http:
  middlewares:
    middlewares-rate-limit:
      rateLimit:
        average: 100
        burst: 50

    middlewares-https-redirectscheme:
      redirectScheme:
        scheme: https
        permanent: true

    middlewares-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlMaxAge: 100
        hostsProxyHeaders:
          - 'Host'
          - 'X-Forwarded-Host'
          - 'X-Forwarded-Proto'
          - 'X-Forwarded-For'
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        forceSTSHeader: true
        customFrameOptionsValue: 'allow-from https:{{env "DOMAINNAME"}}' #CSP takes care of this but may be needed for organizr.
        contentTypeNosniff: true
        browserXssFilter: true
        referrerPolicy: 'same-origin'
        permissionsPolicy: 'camera=(), microphone=(), geolocation=(), payment=(), usb=(), vr=()'
        customResponseHeaders:
          X-Robots-Tag: 'noindex,nofollow' #'none,noarchive,nosnippet,notranslate,noimageindex'
          server: ''

    middlewares-no-frame-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlMaxAge: 100
        hostsProxyHeaders:
          - 'Host'
          - 'X-Forwarded-Host'
          - 'X-Forwarded-Proto'
          - 'X-Forwarded-For'
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        forceSTSHeader: true
        customFrameOptionsValue: 'SAMEORIGIN'
        contentTypeNosniff: true
        browserXssFilter: true
        referrerPolicy: 'same-origin'
        permissionsPolicy: 'camera=(), microphone=(), geolocation=(), payment=(), usb=(), vr=()'
        customResponseHeaders:
          X-Robots-Tag: 'noindex,nofollow'
          server: ''

    middlewares-no-cors-headers:
      headers:
        hostsProxyHeaders:
          - 'Host'
          - 'X-Forwarded-Host'
          - 'X-Forwarded-Proto'
          - 'X-Forwarded-For'
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        forceSTSHeader: true
        customFrameOptionsValue: 'SAMEORIGIN'
        contentTypeNosniff: true
        browserXssFilter: true
        referrerPolicy: 'same-origin'
        permissionsPolicy: 'camera=(), microphone=(), geolocation=(), payment=(), usb=(), vr=()'
        customResponseHeaders:
          X-Robots-Tag: 'noindex,nofollow'
          server: ''

    middlewares-authelia:
      forwardAuth:
        address: "http://authelia:9091/api/verify?rd=https://authelia.{{env "DOMAINNAME"}}"
        trustForwardHeader: true
        authResponseHeaders:
          - 'Remote-User'
          - 'Remote-Groups'
          - 'Remote-Email'
          - 'Remote-Name'

    middlewares-authelia-basic:
      forwardAuth:
        address: "https://authelia:9091/api/verify?auth=basic"
        trustForwardHeader: true
        authResponseHeaders:
          - 'Remote-User'
          - 'Remote-Groups'
          - 'Remote-Email'
          - 'Remote-Name'

    # https://stackoverflow.com/questions/49717670/how-to-config-upload-body-size-restriction-in-traefik
    middlewares-buffering:
      buffering:
        maxRequestBodyBytes: 53687091000
        memRequestBodyBytes: 209715200
        maxResponseBodyBytes: 53687091000
        memResponseBodyBytes: 209715200
        retryExpression: 'IsNetworkError() && Attempts() <= 2'

    middlewares-nextcloud-redirect:
      redirectRegex:
        permanent: true
        regex: "https://(.*)/.well-known/(card|cal)dav"
        replacement: "https://${1}/remote.php/dav/"

http:
  middlewares:
    chain-no-auth:
      chain:
        middlewares:
          - middlewares-rate-limit
          - middlewares-https-redirectscheme
          - middlewares-secure-headers

    chain-authelia:
      chain:
        middlewares:
          - middlewares-rate-limit
          - middlewares-https-redirectscheme
          - middlewares-secure-headers
          - middlewares-authelia

Protonova commented 2 weeks ago

Nothing crazy pops out, I'll take a better look after dinner. However, I noticed that you have:

trustedIPs: &trustedIps
        # Local IP range
        #- 127.0.0.1/32
        #- 10.0.0.0/8
        #- 172.16.0.0/12

I am assuming that this is a local wildcard domain you're using? If so I suggest allowing local network access. If not, confirm that you can ping/nslookup/dig the TA server. Also ensure the path it takes is the correct network hops you expect.

As far as your labels go:

      - traefik.enable=true
      - traefik.http.routers.tubearchivist-rtr.entrypoints=https
      - traefik.http.routers.tubearchivist-rtr.rule=Host(`tubearchivist.$DOMAINNAME`)
      - traefik.http.routers.tubearchivist-rtr.middlewares=chain-authelia@file #Authelia Authentication
      - traefik.http.routers.tubearchivist-rtr.service=tubearchivist-svc
      - traefik.http.services.tubearchivist-svc.loadbalancer.server.port=8000

I would also add (just to ensure the headers are passed through the balancer:

      - traefik.http.services.tubearchivist-svc.loadbalancer.passhostheader=true

tehniemer commented 2 weeks ago

I'm not using Traefik for serving anything locally, it all goes through Cloudflare. Adding the load balancer label didn't change the behavior, and according to the Traefik docs, host headers are passed by default. I bypassed Traefik locally and the companion does connect, so you're right something is up with my reverse proxy.

I've been scouring logs and can't seem to find anywhere that indicates the companion is getting through to Traefik, I'm now wondering if it's getting stopped by Cloudflare.

Protonova commented 1 week ago

Sorry it's been a crazy busy week for me. Have you had any luck in finding the issue? As I suspected it wasn't related to the TA companion. I would say just start troubleshooting 1 by 1.

I would start off with bypassing authelia (domain to bypass).
Comment out 1 middleware at a time.
Move on to looking into if CF is blocking something (maybe start with this 1st? Maybe check if WAF is getting in the way first lol).

MightyDjinn commented 1 week ago

This is due to the proxy intercepting the request to the api (Which uses its own api keys). Add a configuration like this:


  rules:
  - domain: 'app.example.com'
    policy: 'bypass'
    resources:
    - '^/api([/?].*)?$'

tehniemer commented 1 week ago

I have that rule in my Authelia config, which is working for other services that use an API for access, furthermore the behavior here is the same whether Authelia is in the middle or not. I'm unable to see this subdomain pop up in any cloudflare or traefik logs, so I'm a bit stuck at the moment.

tubearchivist / browser-extension

Cannot connect to server error #47