sbernauer commented 8 months ago

Update: This issue has been rescoped from adding labels to all volumes to just adding labels to the listener volumes. This means the scope has been reduced from refactoring all operators to only refactoring the op-rs and the hdfs operator.

Title

Ensure all listener ephemeral volumes have recommended labels.

Description

Currently there are two different ways to attach listener volumes to hdfs pods:

namenodes: use volume claims. These inherit labels from the spec.template of the StatefulSet and are fine.
datanodes: uses volumes attached directly to the pod. These do not inherit the labels of the StatefulSet and are the reason of this issue. Because of this, the endpoints are not listed by stackablectl stacket list.

Update: this was the original description. Keeping it for later reference:

We are missing labels on PVCs created by ephemeral volumes (such as datanode listeners). I fear the only way of fixing this is by putting them on the Statefulset spec.template.spec.volumes[_].ephemeral.volumeClaimTemplate.metadata.lables here. PVCs created by StatefulSet spec.volumeClaimTemplates (such as data or persistent listener volumes) seem to be fine.

This was an oversight in https://github.com/stackabletech/issues/issues/188#issuecomment-1973137082

Value

stackabletctl stacklet list can list both namenodes and datanode endpoints.

Dependencies

none

Tasks

[x] The ListenerOperatorVolumeSourceBuilder in operator-rs ensures that recommended labels attached to the ephemeral and persistent volume explicitely.
[x] The hdfs operator is updated to new operator-rs version.
[x] Every operator that creates listener volumes (currently only hdfs) is checked if it needs to be updated.

Related PRs

Acceptance Criteria

Listener ephemeral and persistent volumes have labels

(Information Security) Risk Assessment

Accessibility Assessment

Quality

Release Notes

No entry

lfrancke commented 5 months ago

I'd like to see a refinement done on this before starting the work according to our template.

sbernauer commented 5 months ago

done