As Stackable is enabling multiple data volumes for hdfs nodes, it would be good to recreate the hdfs cluster with one datanode instance per server with all the volumes attached. Link to PR: https://github.com/stackabletech/hdfs-operator/pull/291 implementing it.
Tasks that needs to be done:
[ ] Update the CRD to contain the new fields
[ ] Update the hdfs-cluster.yaml so it takes x amount of disk per instance with labeling of disk type
[ ] Reduce the overall number of datanodes instances from 32 to 4
[ ] Scale the datanode instances memory and cpu to handle the extra traffic
As Stackable is enabling multiple data volumes for hdfs nodes, it would be good to recreate the hdfs cluster with one datanode instance per server with all the volumes attached. Link to PR: https://github.com/stackabletech/hdfs-operator/pull/291 implementing it.
Tasks that needs to be done: