stackabletech / hdfs-operator

Apache Hadoop HDFS operator for Stackable
Other
40 stars 4 forks source link

fix: Add labels to ephemeral (listener) volumes #534

Closed razvan closed 5 months ago

razvan commented 5 months ago

Description

:green_circle: CI https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hdfs-operator-it-custom/143/

Part of https://github.com/stackabletech/issues/issues/555

Depends on https://github.com/stackabletech/operator-rs/pull/799

Ensure the ephemeral (listener) volumes attached directly to the datanode containers are labeled with the "recommended labels".

This is now the expected structure:

metadata:
  name: hdfs-datanode-default
spec:
  template:
    volumes:
    - ephemeral:
        volumeClaimTemplate:
          metadata:
            annotations:
              listeners.stackable.tech/listener-class: external-unstable
            creationTimestamp: null
            labels:
              app.kubernetes.io/component: datanode
              app.kubernetes.io/instance: hdfs
              app.kubernetes.io/name: hdfs
              app.kubernetes.io/role-group: default
          spec:
            accessModes:
            - ReadWriteMany
            resources:
              requests:
                storage: "1"
            storageClassName: listeners.stackable.tech
            volumeMode: Filesystem
      name: listener

Stacklet listing now includes datanode endpoints as they inherit the labels of the (listener) ephemeral volumes:

❯ stackablectl stacklet list -n kuttl-test-direct-wolf

┌───────────┬─────────┬────────────────────────┬──────────────────────────────────────────────────────────────┬─────────────────────────────────┐
│ PRODUCT   ┆ NAME    ┆ NAMESPACE              ┆ ENDPOINTS                                                    ┆ CONDITIONS                      │
╞═══════════╪═════════╪════════════════════════╪══════════════════════════════════════════════════════════════╪═════════════════════════════════╡
│ hdfs      ┆ hdfs    ┆ kuttl-test-direct-wolf ┆ datanode-default-0-listener-data     172.18.0.2:30580        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-0-listener-http     http://172.18.0.2:32297 ┆                                 │
│           ┆         ┆                        ┆ datanode-default-0-listener-ipc      172.18.0.2:31998        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-0-listener-metrics  172.18.0.2:30805        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-data     172.18.0.2:30990        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-http     http://172.18.0.2:32605 ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-ipc      172.18.0.2:30825        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-metrics  172.18.0.2:30264        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-0-http              http://172.18.0.2:31438 ┆                                 │
│           ┆         ┆                        ┆ namenode-default-0-metrics           172.18.0.2:31633        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-0-rpc               172.18.0.2:31447        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-1-http              http://172.18.0.2:30285 ┆                                 │
│           ┆         ┆                        ┆ namenode-default-1-metrics           172.18.0.2:32287        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-1-rpc               172.18.0.2:30711        ┆                                 │
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ zookeeper ┆ hdfs-zk ┆ kuttl-test-direct-wolf ┆ zk                                   172.18.0.2:32241        ┆ Available, Reconciling, Running │
└───────────┴─────────┴────────────────────────┴──────────────────────────────────────────────────────────────┴─────────────────────────────────┘

Use "stackablectl stacklet credentials [OPTIONS] <PRODUCT_NAME> <STACKLET_NAME>" to display credentials for deployed stacklets.

Definition of Done Checklist

# Author
- [ ] Changes are OpenShift compatible
- [ ] CRD changes approved
- [ ] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
- [ ] Helm chart can be installed and deployed operator works
- [ ] Integration tests passed (for non trivial changes)
- [ ] Changes need to be "offline" compatible
# Reviewer
- [ ] Code contains useful comments
- [ ] Code contains useful logging statements
- [ ] (Integration-)Test cases added
- [ ] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
- [ ] Changelog updated
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)
# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added
- [ ] [Roadmap](https://github.com/orgs/stackabletech/projects/25/views/1) has been updated
sbernauer commented 5 months ago

~I noticed the journalnodes are missing Listeners. As you can not port-forward, there is no way to access the journalnode UI AFAIK. I think we should expose that UI as well, what do you think?~

Was already discussed in Slack, no need to expose the JN UI