apache / incubator-heron

Apache Heron (Incubating) is a realtime, distributed, fault-tolerant stream processing engine from Twitter
https://heron.apache.org/
Apache License 2.0
3.64k stars 598 forks source link

Heron Kubernetes Deployment does not work if Kubernetes is in containerized environment #2500

Closed mjschmidt closed 6 years ago

mjschmidt commented 7 years ago

heron/deploy/kubernetes/general/bookkeeper.yaml

bookkeeper yaml could use some work. Will not work if Kubernetes worker nodes are containerizered due to the way bookkeeeper (apache distributed log) is attempting to port out of the container.

`containers:

    - name: bookie

      image: apachedistributedlog/distributedlog:0.5.0

      resources:

        requests:

          memory: "3Gi"

          cpu: "1000m"

        limits:

          memory: "5Gi"

          cpu: "2000m"

      # use the patched entrypoint.sh - it will automatically created the desired distributedlog namespace

      command: [ "/bin/bash", "/opt/distributedlog/bin/entrypoint.sh" ]

      args: ["/opt/bookkeeper/bin/bookkeeper", "bookie"]

      ports:

        - name: client

          containerPort: 3181

          # we are using `status.hostIP` for the bookie's advertised address. export 3181 as the hostPort,

          # so that the containers are able to access the host port

          hostPort: 3181`
sijie commented 7 years ago

Created apache/bookkeeper#681 for providing a yaml file to run bookkeeper using StatefulSets.