Docs: NetworkPolicies

[genofire]

Is your documentation request related to a feature? If so, which one?

Communication between Pods

Describe the solution that you’d like or the expected outcome

Helm-Chart with Networkpolicies

---

beside a networkpolicy for reach:

• the ingress-controller, the nginx pod
• to reach s3 bucket from:
  • compactor
  • ingester
  • querier
  • store_gateway
  • alertmanager
  • ruler

i start to write some networkpolicies just for ingress:

---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-compactor
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: compactor
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/part-of: memberlist
      ports:
        - port: 7946
          protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-distributor
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: distributor
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/part-of: memberlist
      ports:
        - port: 7946
          protocol: TCP
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: nginx
      ports:
        - port: 8080
          protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-ingester
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: ingester
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/part-of: memberlist
      ports:
        - port: 7946
          protocol: TCP
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: distributor
      ports:
        - port: 9095
          protocol: TCP
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: querier
      ports:
        - port: 9095
          protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-querier
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: querier
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/part-of: memberlist
      ports:
        - port: 7946
          protocol: TCP
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: query-scheduler
      ports:
        - port: 9095
          protocol: TCP
    # if scheduler is disabled:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: query-frontend
      ports:
        - port: 9095
          protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-query-frontend
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: query-frontend
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: nginx
      ports:
        - port: 8080
          protocol: TCP
    # TODO no idea why - maybe send result
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: querier
      ports:
        - port: 9095
          protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-query-scheduler
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: query-scheduler
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: query-frontend
      ports:
        - port: 9095
          protocol: TCP
    # TODO no idea why -  maybe shutdown of querier
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: querier
      ports:
        - port: 9095
          protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mimir-store-gateway
  namespace: phoenix-monitoring-mimir
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/component: store-gateway
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/part-of: memberlist
      ports:
        - port: 7946
          protocol: TCP
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/component: querier
      ports:
        - port: 9095
          protocol: TCP

[dimitarvdimitrov]

a quick note: the alertmanager and ruler might also need to reach the s3 bucket

[genofire]

@dimitarvdimitrov i added in the list above.

i still has problems with the ingester it becomes full without any error messages that it could not upload to s3 ... so i am not sure if i got every needed rule.

EDIT: After create clean PVCs for all ingesters, it look like it works.

[genofire]

PS: Does all component needs to make gossip to all other components? My thought is that the Ring just needed inside one component (or i am wrong?)

[dimitarvdimitrov]

all components need to be able to gossip to all others. All components gossip all rings even if they don't use them.