pravega / pravega-operator

Pravega Kubernetes Operator
Apache License 2.0
41 stars 38 forks source link

Update external access documentation #568

Open mpalczynski opened 3 years ago

mpalczynski commented 3 years ago

Description

Could not expose Pravega cluster in kubernetes using NodePort type described in documentation. Before configuring external access, cluster is up and running. After editing cluster manifest segment store - pod could not startup:

apiVersion: "pravega.pravega.io/v1alpha1"
kind: "PravegaCluster"
metadata:
  name: "example"
spec:
  externalAccess:
    enabled: true
    type: NodePort

  bookkeeper:
    serviceAccountName: pravega-components
...
  pravega:
    controllerServiceAccountName: pravega-components
    segmentStoreServiceAccountName: pravega-components
...

Problematic pods

pod/pravega-pravega-pravega-controller-75fd74c5f5-knmgb   0/1     Running            0          68m
pod/pravega-pravega-pravega-segment-store-0               0/1     CrashLoopBackOff   7          13m

Node Port service has been created for segment store. However pravega controller node port is missing:

service/pravega-pravega-pravega-segment-store-0 NodePort 10.101.73.80 <none> 12345:30746/TCP

Logs:

kubectl logs -f pravega-pravega-pravega-segment-store-0
Waiting for pravega-bookkeeper-bookie-headless:3181
Connected to pravega-bookkeeper-bookie-headless:3181
Running in a Kubernetes environment and managed by the Pravega Operator with external access enabled
Failed to get External Service. Exiting...

I think it is something related to service account pravega-components not linked to pods through Pravega Cluster manifest. After executing kubernetes API call: curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer ..." https://kubernetes.default.svc/api/v1/namespaces/example/services/pravega-pravega-pravega-segment-store-0 I got an error:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "services \"pravega-pravega-pravega-segment-store-0\" is forbidden: User \"system:serviceaccount:example:default\" cannot get resource \"services\" in API group \"\" in the namespace \"example\"",
  "reason": "Forbidden",
  "details": {
    "name": "pravega-pravega-pravega-segment-store-0",
    "kind": "services"
  },
  "code": 403
}

It seems controllerServiceAccountName, segmentStoreServiceAccountName are not taken into account since those pods have default service account assigned.

Pravega operator version: 0.5.3 Pravega version: 0.9.0

Importance

should-have

Location

documentation

Suggestions for an improvement

Update aforementioned documentation adding description of NodePort type of external access.

anishakj commented 3 years ago

@mpalczynski Could you please point us to the documentation you are referring?

mpalczynski commented 3 years ago

@anishakj https://github.com/pravega/pravega-operator/blob/master/doc/external-access.md

anishakj commented 3 years ago

@mpalczynski , We have realized that we are not supporting node port, and there is a mismatch between documentation of external access. Would you like to contribute in providing the support for node port