Usage monitor not receiving data

[AdrianHordyk]

I followed this guide to get my app running with Shiny Proxy, Traefik, and Docker Swarm. Everything is working great.

I attempted to follow these instructions to monitor usage statistics using Micrometer, Prometheus, and Grafana.

I'm able to access Granafa from my domain name, and set the data source to http://prometheus:7070. Save & Test shows it's able to connect to the data source.

The problem is the ShinyProxy dashboard receives no data. I get errors like "1:68: parse error: missing unit character in duration" which suggests to me that no data from ShinyProxy is available.

I suspect the issue is because I'm running ShinyProxy and Prometheus/Grafana in separate Docker stacks and that the issue may be in:

   static_configs:
      - targets: ['localhost:9090'] 

I've tried different approaches I've found online, but there's very little information available on monitoring usage other than the official instructions.

I'm hoping someone can point out an obvious place where I've gone wrong.

This is my shinyproxy.yml:

version: '3.3'

services:

  shinyproxy:
    image: presstofan/shinyproxy-example
    ports:
      - 8080
    networks:
      - traefik-public
      - sp-net
    deploy:
      replicas: 1
      restart_policy:
        condition: on-failure
      placement:
        constraints:
          - node.role==manager
      labels:
          - traefik.enable=true # enable traefik
          - traefik.docker.network=traefik-public # put it in the same network as traefik
          - traefik.constraint-label=traefik-public # assign the same label as traefik so it can be discovered
          - traefik.http.routers.shinyproxy.rule=Host(`${APP_DOMAIN?Variable not set}`) # listen to port 80 for request to APP_DOMAIN (use together with the line below)
          - traefik.http.routers.shinyproxy.entrypoints=http
          - traefik.http.middlewares.shinyproxy.redirectscheme.scheme=https # redirect traffic to https
          - traefik.http.middlewares.shinyproxy.redirectscheme.permanent=true # redirect traffic to https
          - traefik.http.routers.shinyproxy-secured.rule=Host(`${APP_DOMAIN?Variable not set}`) # listen to port 443 for request to APP_DOMAIN (use together with the line below)
          - traefik.http.routers.shinyproxy-secured.entrypoints=https
          - traefik.http.routers.shinyproxy-secured.tls.certresolver=le # use the Let's Encrypt certificate we set up earlier
          - traefik.http.services.shinyproxy-secured.loadbalancer.server.port=8080 # ask Traefik to search for port 8080 of the shinyproxy service container
    volumes:
      - ./application/application.yml:/opt/shinyproxy/application.yml
      - /var/run/docker.sock:/var/run/docker.sock

  myapp:
    image: adrianrh/myapp
    networks:
      - sp-net

networks:
  traefik-public:
    external: true
  sp-net:
    external: true

application.yml:

proxy:
  title: My App
  landing-page: /app/myapp
  port: 8080 # use Port 8080 for ShinyProxy

  container-wait-time: 60000 
  heartbeat-rate: 10000 
  heartbeat-timeout: 60000 

  usage-stats-url: micrometer

  management:
    metrics:
      export:
        prometheus:
          enabled: true

  container-backend: docker-swarm

  docker:
      internal-networking: true

  specs:
    - id: myapp
      display-name: My App
      container-cmd: ["R", "-e", "shiny::runApp('/root/myapp', host='0.0.0.0', port=3838)"]
      container-image: adrianrh/myapp
      container-network: sp-net

server:
  useForwardHeaders: true 
  servlet.session.timeout: 0

logging:
  file:
    shinyproxy.log

usagestats.yml:

version: '3.3'

services:

  grafana:
    image: grafana/grafana:7.0.1
    ports:
      - 3000
    networks:
      - traefik-public
    environment:
      - GF_SECURITY_ADMIN_USER=${ADMIN_USER:-admin}
      - GF_SECURITY_ADMIN_PASSWORD=${ADMIN_PASSWORD:-admin}
      - GF_USERS_ALLOW_SIGN_UP=false
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == manager
      resources:
        limits:
          memory: 128M
        reservations:
          memory: 64M
      labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.http.routers.grafana.rule=Host(`${GRAFANA_DOMAIN?Variable not set}`)
          - traefik.http.routers.grafana.entrypoints=http
          - traefik.http.middlewares.grafana.redirectscheme.scheme=https
          - traefik.http.middlewares.grafana.redirectscheme.permanent=true
          - traefik.http.routers.grafana-secured.rule=Host(`${GRAFANA_DOMAIN?Variable not set}`)
          - traefik.http.routers.grafana-secured.entrypoints=https
          - traefik.http.routers.grafana-secured.tls.certresolver=le
          - traefik.http.services.grafana-secured.loadbalancer.server.port=3000
    volumes:
      - grafana:/var/lib/grafana

  prometheus:
    image: prom/prometheus
    ports: 
      - 7070
    networks:
      - traefik-public
      - sp-net
    deploy:
      resources:
        reservations:
          cpus: '0.3'
          memory: 128M
        limits:
          cpus: '0.6'
          memory: 512M
      placement:
        constraints:
          - node.role==manager
    volumes:
      - ./prometheus:/etc/prometheus
      - prometheus-data:/prometheus
    command: 
      - '--web.enable-lifecycle'
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--web.listen-address=:7070'

networks:
  traefik-public:
    external: true
  sp-net:
    external: true

volumes:
  grafana:
  prometheus-data:

and prometheus.yml:

scrape_configs:
  - job_name: 'shinyproxy'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['localhost:9090'] 

[AdrianHordyk]

Update: making some progress but target state is DOWN

prometheus.yml now set to to:

scrape_configs:
  - job_name: 'shinyproxy'
    metrics_path: '/actuator/prometheus'
    static_configs:
      - targets: ['shinyproxy:9090'] 

Also, I realized I had the usage management section in the wrong place in application.yml (was inside proxy section previously).

Prometheus is now accessible via domain name (made some other changes to usagestats.yml for this) and shows this:

Endpoint: http://shinyproxy:9090/actuator/prometheus State: DOWN Error: Get "http://shinyproxy:9090/actuator/prometheus": dial tcp 10.0.2.48:9090: connect: connection refused

Do this mean ShinyProxy is not serving the usage stats to port 9090 or an error with the docker containers communicating with each other?

[AdrianHordyk]

Found the issue: it was something stupid of course.

The shinyproxy docker image I was using (presstofan/shinyproxy-example) was an old version where usage stats monitoring with micrometer etc hadn't been implemented yet.

For anyone else following the databentobox.com guide, be aware that the latest version of shinyproxy requires a few modifications to the configuration.