stefanprodan / dockprom

Docker hosts and containers monitoring with Prometheus, Grafana, cAdvisor, NodeExporter and AlertManager
MIT License
6.05k stars 1.73k forks source link

Container with network_mode: host is being tagged with network throughput from a different container #229

Closed 40Cakes closed 3 years ago

40Cakes commented 3 years ago

Hey guys,

I have a container that is configured with network_mode: host to get the host's IPv6 address, I just noticed that this container is being tagged with most of the network throughput on the system when in reality this container uses barely any resources. It should be mullvad-vpn container since this spike was when I tested a FreeBSD ISO download via qBittorrent (which uses the mullvad-vpn container as a service (see below)).

image

Can I label certain containers to disregard them from monitoring? Or is there a way to resolve this incorrect network throughput reporting?

Excerpt of my docker compose file as reference if required: (I can post the whole thing if required, but it's almost 1,000 lines long 😅 )

  # CloudFlare Dynamic DNS IPv6
  # https://github.com/oznu/docker-cloudflare-ddns
  # https://hub.docker.com/r/oznu/cloudflare-ddns/
  cloudflare-ddns-ipv6:
    container_name: cloudflare-ddns-ipv6
    image: oznu/cloudflare-ddns:latest
    restart: unless-stopped
    network_mode: host
    security_opt:
      - no-new-privileges:true
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - API_KEY=${CLOUDFLARE_DDNS_API_KEY}
      - ZONE=${DOMAINNAME}
      - PROXIED=true
      - RRTYPE=AAAA
      - DNS_SERVER=${DNS_SERVER}
      - INTERFACE=${HOST_INTERFACE}
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}

  # Wireguard/Mullvad VPN - simple yet fast and modern VPN that utilizes state-of-the-art cryptography
  # https://hub.docker.com/r/linuxserver/wireguard
  # https://github.com/WireGuard/wireguard-linux
  # https://mullvad.net/
  mullvad-vpn:
    image: linuxserver/wireguard:latest
    container_name: mullvad-vpn
    restart: unless-stopped
    networks:
      internal:
        aliases: 
          - qbittorrent
          - jackett
    ports:
      - 9117:9117     # Jackett
      - 6969:6969     # qBittorrent
      - 6881:6881     # qBittorrent
      - 6881:6881/udp # qBittorrent
    volumes:
      - ${DOCKERDIR}/wireguard/config:/config:rw
      - /lib/modules:/lib/modules:ro
      - /etc/localtime:/etc/localtime:ro
      - ${DOCKERDIR}/shared:/shared:rw
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
      - net.ipv6.conf.all.disable_ipv6=0
    labels:
      - traefik.enable=true
      - traefik.http.routers.qbittorrent.entrypoints=https
      - traefik.http.routers.qbittorrent.rule=Host(`qbittorrent.${DOMAINNAME}`)
      - traefik.http.routers.qbittorrent.middlewares=oauth
      - traefik.http.routers.qbittorrent.service=qbittorrent
      - traefik.http.middlewares.qbittorrent.forwardauth.address=http://oauth:4181
      - traefik.http.middlewares.qbittorrent.forwardauth.authResponseHeaders=X-Forwarded-User
      - traefik.http.middlewares.qbittorrent.forwardauth.trustForwardHeader=true
      - traefik.http.services.qbittorrent.loadbalancer.server.port=6969
      - traefik.http.routers.jackett.entrypoints=https
      - traefik.http.routers.jackett.rule=Host(`jackett.${DOMAINNAME}`)
      - traefik.http.routers.jackett.middlewares=oauth
      - traefik.http.routers.jackett.service=jackett
      - traefik.http.middlewares.jackett.forwardauth.address=http://oauth:4181
      - traefik.http.middlewares.jackett.forwardauth.authResponseHeaders=X-Forwarded-User
      - traefik.http.middlewares.jackett.forwardauth.trustForwardHeader=true
      - traefik.http.services.jackett.loadbalancer.server.port=9117

  # qBittorrent - open-source BitTorrent client
  # https://hub.docker.com/r/linuxserver/qbittorrent
  # https://github.com/qbittorrent/qBittorrent
  qbittorrent:
    container_name: qbittorrent
    image: linuxserver/qbittorrent:latest
    restart: unless-stopped
    depends_on:
      - mullvad-vpn
    network_mode: service:mullvad-vpn
    volumes:
      - ${DOCKERDIR}/qbittorrent/config:/config:rw
      - /tmp/downloads:/downloads:rw # TODO
      - /etc/localtime:/etc/localtime:ro
      - ${DOCKERDIR}/shared:/shared:rw
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
      - UMASK=002
      - WEBUI_PORT=6969

  # Jackett - a proxy server: it translates queries from apps (Sonarr, Radarr, qBittorrent etc.) into tracker-site-specific http queries, parses the html response, then sends results back to the requesting software
  # https://hub.docker.com/r/linuxserver/jackett
  # https://github.com/Jackett/Jackett
  jackett:
    container_name: jackett
    image: linuxserver/jackett:latest
    restart: unless-stopped
    depends_on:
      - mullvad-vpn
    network_mode: service:mullvad-vpn
    volumes:
      - ${DOCKERDIR}/jackett/config:/config:rw
      - ${DOCKERDIR}/jackett/downloads:/downloads:rw
      - /etc/localtime:/etc/localtime:ro
      - ${DOCKERDIR}/shared:/shared:rw
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TZ}
40Cakes commented 3 years ago

Kill cloudflare-ddns-ipv6, queue a download and reporting looks good:

image

Resume the container, and the reporting is borked again:

image
nightah commented 3 years ago

It is reporting correctly, because your network_mode is set to host it will pick up all traffic for the host from that specific container.

You can modify the query to disregard that specific container if you prefer:

Container Network Input: sum by (name) (rate(container_network_receive_bytes_total{name!="cloudflare-ddns-ipv6",image!="",container_label_org_label_schema_group=""}[1m]))

Container Network Output: sum by (name) (rate(container_network_transmit_bytes_total{name!="cloudflare-ddns-ipv6",image!="",container_label_org_label_schema_group=""}[1m]))