matomo-org / docker

Official Docker project for Matomo Analytics
https://matomo.org
Other
841 stars 348 forks source link

Working example Kubernetes config #219

Open jhughes2112 opened 4 years ago

jhughes2112 commented 4 years ago

I had to piece together from the very weak Docker and Docker Compose examples (that are out of date and slightly incorrect/minimal), but got a proper install of this running today. Note, this config is running with Istio generating the HTTPS certificates and handling termination, so all hosting is done via port 80, as is typical behind a load balancer or k8s cluster. Here's the config, do whatever you will with it:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: analytics
  labels:
    app: analytics
spec:
  hosts:
  - "analytics.yourdomainname.com"
  gateways:
  - istio-gw
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: analytics  # this refers to a Service with name="analytics"
        port:
          number: 80
---
apiVersion: v1  # this one is exposed by the Istio VirtualService above
kind: Service
metadata:
  name: analytics
  labels:
    app: analytics
spec:
  ports:
  - port: 80
    name: http-web  
    protocol: TCP
    targetPort: http-web
  selector:
    app: analytics  # send traffic to the analytics pods
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: analytics
  labels:
    app: analytics
spec:
  serviceName: analytics
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  selector:
    matchLabels:
      app: analytics
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "false"  # if this is true, https gets proxied and everything breaks
      labels:
        app: analytics
        version: 1.0.0
    spec:
      containers:
      - image: matomo:fpm-alpine
        imagePullPolicy: Always
        name: analytics
        env:
        - name: MATOMO_DATABASE_ADAPTER
          value: mysql
        - name: MATOMO_DATABASE_HOST
          value: mysql
        - name: MATOMO_DATABASE_TABLES_PREFIX
          value: analytics
        - name: MATOMO_DATABASE_USERNAME
          value: youruser
        - name: MATOMO_DATABASE_PASSWORD
          value: yourpass
        - name: MATOMO_DATABASE_DBNAME
          value: analytics
        resources:
          limits:
            cpu: "3.0"
            memory: "3Gi"
          requests:
            cpu: "0.25"
            memory: "250Mi"
        volumeMounts:
        - name: analytics-storage
          mountPath: /var/www/html
      - image: nginx:latest
        name: nginx
        volumeMounts:
        - name: analytics-storage
          mountPath: /var/www/html
          readOnly: true
        - name: analytics-configmap
          mountPath: /etc/nginx/conf.d/default.conf
          subPath: nginx.conf
        ports:
        - containerPort: 80
          name: http-web
        resources:
          limits:
            cpu: "3.0"
            memory: "3Gi"
          requests:
            cpu: "0.25"
            memory: "250Mi"
      - image: mysql:5.7
        name: mysql
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: yourrootpw
        - name: MYSQL_DATABASE
          value: analytics
        - name: MYSQL_USER
          value: youruser
        - name: MYSQL_PASSWORD
          value: yourpass
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: analytics-db
          mountPath: /var/lib/mysql
        - name: analytics-configmap
          mountPath: /etc/mysql/conf.d/more.cnf
          subPath: more.cnf
        livenessProbe: 
          initialDelaySeconds: 10
          timeoutSeconds: 10
          periodSeconds: 30
          failureThreshold: 5
          tcpSocket:
            port: mysql
        readinessProbe: 
          initialDelaySeconds: 20
          timeoutSeconds: 10
          periodSeconds: 30
          failureThreshold: 5
          tcpSocket:
            port: mysql
        resources:
          limits:
            cpu: "3.0"
            memory: "3Gi"
          requests:
            cpu: "0.25"
            memory: "250Mi"
      volumes:
      - name: analytics-storage
        persistentVolumeClaim:
          claimName: analytics-storage
      - name: analytics-db
        persistentVolumeClaim:
          claimName: analytics-db
      - name: analytics-configmap
        configMap:
          name: analytics-configmap
          items:
          - key: nginx.conf
            path: nginx.conf
          - key: more.cnf
            path: more.cnf
  volumeClaimTemplates:  # this automatically allocates storage via your-storage-class (NFS?)
  - metadata:
      name: analytics-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "your-storage-class"
      resources:
        requests:
          storage: 10Gi
  - metadata:
      name: analytics-db
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "your-storage-class"
      resources:
        requests:
          storage: 20Gi
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: analytics-configmap
  labels:
    app: analytics
data:
  more.cnf: |
    [mysqld]
    innodb_buffer_pool_size=256M
    max_allowed_packet=64M
    sql_mode=STRICT_ALL_TABLES
    wait_timeout = 28800
    interactive_timeout = 28800
  nginx.conf: |
    server {
        listen [::]:80; # remove this if you don't want Matomo to be reachable from IPv6
        listen 80;
        server_name localhost;
        access_log /dev/stdout;
        error_log /dev/stdout;

        add_header Referrer-Policy origin always; # make sure outgoing links don't show the URL to the Matomo instance
        add_header X-Content-Type-Options "nosniff" always;
        add_header X-XSS-Protection "1; mode=block" always;

        root /var/www/html/;

        index index.php;

        ## Begin - Index
        ## only allow accessing the following php files
        location ~ ^/(index|matomo|piwik|js/index|plugins/HeatmapSessionRecording/configs)\.php {
            # Choose either a socket or TCP/IP address
            # fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
            # fastcgi_pass unix:/var/run/php5-fpm.sock; #legacy
            fastcgi_pass localhost:9000;

            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
            fastcgi_param HTTP_PROXY ""; # prohibit httpoxy: https://httpoxy.org/
        }

        ## deny access to all other .php files
        location ~* ^.+\.php$ {
            deny all;
            return 403;
        }

        ## serve all other files normally
        location / {
            try_files $uri $uri/ =404;
        }

        ## disable all access to the following directories
        location ~ ^/(config|tmp|core|lang) {
            deny all;
            return 403; # replace with 404 to not show these directories exist
        }

        location ~ /\.ht {
            deny  all;
            return 403;
        }

        location ~ js/container_.*_preview\.js$ {
            expires off;
            add_header Cache-Control 'private, no-cache, no-store';
        }

        location ~ \.(gif|ico|jpg|png|svg|js|css|htm|html|mp3|mp4|wav|ogg|avi|ttf|eot|woff|woff2|json)$ {
            allow all;
            ## Cache images,CSS,JS and webfonts for an hour
            ## Increasing the duration may improve the load-time, but may cause old files to show after an Matomo upgrade
            expires 1h;
            add_header Pragma public;
            add_header Cache-Control "public";
        }

        location ~ ^/(libs|vendor|plugins|misc/user|node_modules) {
            deny all;
            return 403;
        }

        ## properly display textfiles in root directory
        location ~/(.*\.md|LEGALNOTICE|LICENSE) {
            default_type text/plain;
        }
    }
    # vim: filetype=nginx
romulus-ai commented 3 years ago

A Nice Configuration and for sure a good startingpoint for everyone starting deploying matomo to K8S. I ended up with more or less the same configuration, however used a deployment instead an STS. However a hint from my side! Insert a configmap to increase the memory limit of php-fpm in the matomo container and to increase the max_execution_time. Those values are important if the importers should be used! Additionally you could use the K8S Kind "Cronjob" to run the archiver periodically.

Here my deployment, configmap and cronjob:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: matomo
  labels:
    app: matomo
    environment: matomo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: matomo
      environment: matomo
  template:
    metadata:
      labels:
        app: matomo
        environment: matomo
    spec:
      containers:
      - name: matomo-nginx
        image: "nginx:alpine"
        command: ["nginx", "-g", "daemon off;", "-c", "/config/nginx.conf"]
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 512Gi
        ports:
          - name: http
            containerPort: 80
            protocol: TCP
        volumeMounts:
          - mountPath: /config/nginx.conf
            name: config
            subPath: nginx.conf
          - mountPath: /var/www/html
            name: html-files
      - name: matomo
        image: matomo:3.14.1-fpm-alpine
        env:
        - name: MATOMO_DATABASE_HOST
          value: mysql
        - name: MATOMO_DATABASE_USERNAME
          value: matomo
        - name: MATOMO_DATABASE_PASSWORD
          value: password
        - name: MATOMO_DATABASE_DBNAME
          value: matomo
        - name: MATOMO_DATABASE_ADAPTER
          value: mysql
        - name: MATOMO_DATABASE_TABLES_PREFIX
          value: prefix
        resources:
          requests:
            cpu: 1000m
            memory: 2Gi
          limits:
            cpu: 1000m
            memory: 2Gi
        ports:
        - name: http
          containerPort: 9000
        volumeMounts:
        - mountPath: /usr/local/etc/php/conf.d/php-special.ini
          name: config
          subPath: php.ini
        - mountPath: /var/www/html
          name: html-files

      volumes:
      - name: config
        configMap:
          name: matomo-config
      - name: html-files
        cephfs:
          monitors:
          - 192.168.0.1
          path: /matomo
          user: admin
          secretRef:
            name: ceph-secret
          readOnly: false
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: matomo-config
  labels:
    app: matomo
    environment: matomo
data:
  nginx.conf: |
    # nginx.conf for php
    events {
      worker_connections 768;
    }
    http {
      upstream backend {
        server localhost:9000;
      }
      include /etc/nginx/mime.types;
      default_type application/octet-stream;
      gzip on;
      gzip_disable "msie6";
      # If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
      # scheme used to connect to this server
      map $http_x_forwarded_proto $proxy_x_forwarded_proto {
        default $http_x_forwarded_proto;
        ''      $scheme;
      }
      # If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
      # server port the client connected to
      map $http_x_forwarded_port $proxy_x_forwarded_port {
        default $http_x_forwarded_port;
        ''      $server_port;
      }
      # If we receive Upgrade, set Connection to "upgrade"; otherwise, delete any
      # Connection header that may have been passed to this server
      map $http_upgrade $proxy_connection {
        default upgrade;
        '' close;
      }
      # Set appropriate X-Forwarded-Ssl header
      map $scheme $proxy_x_forwarded_ssl {
        default off;
        https on;
      }
      # HTTP 1.1 support
      proxy_http_version 1.1;
      proxy_buffering off;
      proxy_set_header Host $http_host;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection $proxy_connection;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
      proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
      proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
      proxy_set_header Proxy "";
      server {
        listen 80;
        root /var/www/html/;
        index index.php index.html index.htm;
        location / {
          try_files $uri $uri/ =404;
        }
        error_page 404 /404.html;
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
          root /usr/share/nginx/html;
        }
        location = /favicon.ico {
          log_not_found off;
          access_log off;
        }
        location ~ \.php$ {
          fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
          fastcgi_param  SERVER_SOFTWARE    nginx;
          fastcgi_param  QUERY_STRING       $query_string;
          fastcgi_param  REQUEST_METHOD     $request_method;
          fastcgi_param  CONTENT_TYPE       $content_type;
          fastcgi_param  CONTENT_LENGTH     $content_length;
          fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
          fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
          fastcgi_param  REQUEST_URI        $request_uri;
          fastcgi_param  DOCUMENT_URI       $document_uri;
          fastcgi_param  DOCUMENT_ROOT      $document_root;
          fastcgi_param  SERVER_PROTOCOL    $server_protocol;
          fastcgi_param  REMOTE_ADDR        $remote_addr;
          fastcgi_param  REMOTE_PORT        $remote_port;
          fastcgi_param  SERVER_ADDR        $server_addr;
          fastcgi_param  SERVER_PORT        $server_port;
          fastcgi_param  SERVER_NAME        $server_name;
          fastcgi_intercept_errors on;
          fastcgi_pass backend;
        }
      }
    }
  php.ini: |
    memory_limit=1536M
    max_execution_time=1440

---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: matomo-archive
spec:
  schedule: "*/30 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: matomo-archive
            image: matomo:3.14.1-fpm-alpine
            command: ["/usr/local/bin/php"]
            args: ["/var/www/html/console", "core:archive", "--url=https://matomo.example.com"]
            env:
            - name: MATOMO_DATABASE_HOST
              value: mysql
            - name: MATOMO_DATABASE_USERNAME
              value: matomo
            - name: MATOMO_DATABASE_PASSWORD
              value: password
            - name: MATOMO_DATABASE_DBNAME
              value: matomo
            - name: MATOMO_DATABASE_ADAPTER
              value: mysql
            - name: MATOMO_DATABASE_TABLES_PREFIX
              value: prefix
            resources:
              requests:
                cpu: 100m
                memory: 512Mi
              limits:
                cpu: 500m
                memory: 1Gi
            volumeMounts:
            - mountPath: /var/www/html
              name: html-files
          restartPolicy: OnFailure
          volumes:
          - name: html-files
            cephfs:
              monitors:
              - 192.168.0.1
              path: /matomo
              user: admin
              secretRef:
                name: ceph-secret
              readOnly: false
jhughes2112 commented 3 years ago

Very helpful! Thanks for the note.

javierguzman commented 3 years ago

Thanks! Just starting with Matomo and trying to deploy on K8s. A couple of questions; What is html-files used for? And what does the archiver do?

I believe this does not use Apache as I see nginx, does it?

dschuldt commented 3 years ago

Great example!

Although your php setting _ memorylimit=1536M does not make sense since you limit your pod to 1Gi.

Choko256 commented 2 years ago

I'm not sure to understand... you are adding a custom nginx here. So at the end with the need of an ingress to expose the Matomo service, and supposing I'd want to use a cert manager to enable HTTPS, each request will be proxified along these :

[ Cert Manager ] -> [ Nginx Controller Ingress ] -> [ Exposing Ingress ] -> [ Nginx Service with custom configuration ] -> [ PHP-FPM ]

Am I right ? Because it seems a lot of proxifying for just a PHP app !?

dasuchin commented 2 years ago

I'm trying to get this running in EKS right now, and using similar configs, although I'm using a helm chart to manage it. I keep getting a 504 or 502 when I try to access the plugins page, and in general the entire application is sluggish. Is that to be expected? Or is that why you have the resources cranked up so high in the examples here?

It doesn't look like it's maxing out the resources on my install, so I'm not sure if that's the solve here or not.

warent commented 2 years ago

@Choko256 that sounds right. This is the kind of thing that naturally occurs when using Docker or especially when using Kubernetes. In theory all those proxy/ingress layers should be basically free

romulus-ai commented 2 years ago

@Choko256 yes, thats the way it is in the container world, sounds weird, however it makes debugging and tracing much easier, since the component going crazy is easier to detect.

@dasuchin may the 500 errors appear because the upstream to matomo is not working, here the problem could either be, that matomo is crashing all the time because the resource limit is reached (you can check this by looking for the uptime of the container) or maybe you have some kind of networkrules blocking here. However, the resources I used in the example, were really necessary for a quite simple test setup, so may increase your ressource limits.

@dschuldt the memorylimit of the matomo container is 2Gi, 1Gi is the limit of the crontab container, which is not using php.ini