search

splunk / splunk-operator

Splunk Operator for Kubernetes
Other
209 stars 115 forks source link

Add example of ingress documentation for ALB #16

Open ghost opened 5 years ago

ghost commented 5 years ago

this example use amazon ALB to minimize costs setup include HTTP/2 for performance SSL redirection and TLS offload using amazon certs

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "splunk"
  labels:
    app: splunk
    role: http
    tier: ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/tags: Environment=dev,Team=test
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:174701313045:certificate/57dfd87e-f18b-426b-b0a8-a10a38a6df54
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
#    external-dns.alpha.kubernetes.io/hostname:  master.mycluster.example.com,license.mycluster.example.com,sh-dev.mycluster.example.com,inputs-hec.mycluster.example.com
spec:
  rules:
    - host: master.mycluster.example.com
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-cluster-master-service-public
              servicePort: 8000
    - host: license.mycluster.example.com
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-license-master-service-public
              servicePort: 8000
    - host: splunk.mycluster.example.com
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-search-head-service-public
              servicePort: 8000
    - host: inputs-hec.mycluster.example.com
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-indexer-public
              servicePort: 8088
---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-cluster-master-service-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
spec:
  selector:
    app: splunk
    for: dfscluster
    type: cluster-master
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-license-master-service-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
     alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
     alb.ingress.kubernetes.io/healthy-threshold-count: 10
spec:
  selector:
    app: splunk
    for: dfscluster
    type: license-master
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-search-head-service-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /en-US/account/login
     #alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
     alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
     alb.ingress.kubernetes.io/healthy-threshold-count: 10
     alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=600
     alb.ingress.kubernetes.io/load-balancer-attributes:routing: http2.enabled=true
spec:
  selector:
    app: splunk
    for: dfscluster
    type: search-head
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-indexer-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /services/collector/health
     alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
     alb.ingress.kubernetes.io/healthy-threshold-count: 10
     service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
     service.beta.kubernetes.io/do-loadbalancer-algorithm: "least_connections"
spec:
  selector:
    app: splunk
    for: dfscluster
    type: indexer
  ports:
    - protocol: TCP
      port: 8088
      targetPort: 8088
  type: LoadBalancer
ryanfaircloth commented 5 years ago

Updated example which avoid provisioning ELBs

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: "splunk"
  labels:
    app: splunk
    role: http
    tier: ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/tags: Environment=dev,Team=test
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:174701313045:certificate/57dfd87e-f18b-426b-b0a8-a10a38a6df54
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=600
    alb.ingress.kubernetes.io/load-balancer-attributes: routing.http2.enabled=true

#    external-dns.alpha.kubernetes.io/hostname:  master.kops.spl.guru,license.kops.spl.guru,sh-dev.kops.spl.guru,inputs-hec.kops.spl.guru
spec:
  rules:
    - host: master.kops.spl.guru
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-cluster-master-service-public
              servicePort: 8000
    - host: license.kops.spl.guru
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-license-master-service-public
              servicePort: 8000
    - host: splunk.kops.spl.guru
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-search-head-service-public
              servicePort: 8000
    - host: inputs-hec.kops.spl.guru
      http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: splunk-dfscluster-indexer-public
              servicePort: 8088

---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-cluster-master-service-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
spec:
  selector:
    app: splunk
    for: dfscluster
    type: cluster-master
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-license-master-service-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
     alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
     alb.ingress.kubernetes.io/healthy-threshold-count: 10
spec:
  selector:
    app: splunk
    for: dfscluster
    type: license-master
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: NodePort

---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-search-head-service-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /en-US/account/login
     #alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
     alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
     alb.ingress.kubernetes.io/healthy-threshold-count: 10
     alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=600
     alb.ingress.kubernetes.io/load-balancer-attributes: routing.http2.enabled=true
spec:
  selector:
    app: splunk
    for: dfscluster
    type: search-head
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: splunk-dfscluster-indexer-public
  annotations:
     alb.ingress.kubernetes.io/healthcheck-path: /services/collector/health
     alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
     alb.ingress.kubernetes.io/healthy-threshold-count: 10
     service.beta.kubernetes.io/do-loadbalancer-protocol: "https"
     service.beta.kubernetes.io/do-loadbalancer-algorithm: "least_connections"
     alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=false,stickiness.lb_cookie.duration_seconds=60
spec:
  selector:
    app: splunk
    for: dfscluster
    type: indexer
  ports:
    - protocol: TCP
      port: 8088
      targetPort: 8088
  type: NodePort
mikedickey commented 4 years ago

Adding for reference: https://aws.amazon.com/blogs/containers/using-alb-ingress-controller-with-amazon-eks-on-fargate/

mikedickey commented 4 years ago

Also TODO: update for new serviceTemplate parameter in 0.1.0.. Something like:

spec:
  serviceTemplate:
    metadata:
      annotations:
        alb.ingress.kubernetes.io/healthcheck-path: /robots.txt
        alb.ingress.kubernetes.io/healthcheck-interval-seconds: 1
        alb.ingress.kubernetes.io/healthy-threshold-count: 10
    spec:
      type: NodePort
Caldas commented 4 years ago

I would love to change my current ELB approach to this one as soon it's ready 🚀

jstewart612 commented 1 year ago

This issue impacts my present deployment engagement with Splunk because the serviceTemplate field of the CRD does NOT respect metadata.annotations which means I cannot use Azure Kubernetes Service ILBs, so it's not just AWS ALB: it's any cloud Kubernetes provider which uses annotations to customize LoadBalancer services in kubernetes, which is basically all of them.

Your documentation gives examples of wiring through other Ingress controllers like Istio or Nginx. You make me, for most components, terminate SSL end-to-end right at the component. So, I don't want a heavy service mesh config for your components, since you make me terminate SSL at you anyway, I gain almost nothing from this. So, please get serviceTemplate.metadata.annotations working on your CRDs (serviceTemplate.type and serviceTemplate.loadBalancerIP already do work).

adamrushuk commented 1 year ago

This issue impacts my present deployment engagement with Splunk because the serviceTemplate field of the CRD does NOT respect metadata.annotations which means I cannot use Azure Kubernetes Service ILBs, so it's not just AWS ALB: it's any cloud Kubernetes provider which uses annotations to customize LoadBalancer services in kubernetes, which is basically all of them.

Your documentation gives examples of wiring through other Ingress controllers like Istio or Nginx. You make me, for most components, terminate SSL end-to-end right at the component. So, I don't want a heavy service mesh config for your components, since you make me terminate SSL at you anyway, I gain almost nothing from this. So, please get serviceTemplate.metadata.annotations working on your CRDs (serviceTemplate.type and serviceTemplate.loadBalancerIP already do work).

@jstewart612 a workaround for adding annotations was posted here: https://github.com/splunk/splunk-operator/issues/1091#issuecomment-1447091838