Missing variables for service port and firewall configuration

[KaidoKopli]

Installing TDP lib on freshly imaged machine we noticed that all firewall ports were not properly open by the firewalld playbook (ansible_collection/tosit/tdp_extra/playbooks/meta/firewall.yml).

Comparing with the pre TDP lib deployment method we noted the following, ports and port ranges were missing from tdp-cluster.yml

hdfs_nn: [9820] dfs.namenode.rpc-address hdfs_dn: [1004] dfs.datanode.address hive_s2: [10002] hive. server2. webui.port

We suggest that variables are added to the playbook for services listening to these ports as well as variables for the missing port ranges.

New variables for services should be used in the corresponding playbooks instead of hardcoded port values and the firewalld playbook should make sure network flows are open for the services and port ranges.

Is there some plan to add livy variables in the tdp-cluster.yml?

[GuillaumeHold]

This is probably a configuration issue.The properties are defined in tdp-collection as follow :

• "dfs.namenode.rpc-address.{{ cluster_name }}.nn1": "{{ groups['hdfs_nn'][0] | tosit.tdp.access_fqdn(hostvars) }}:{{ hdfs_nn_rpc_port }}"
• dfs.datanode.address: "0.0.0.0:{{ hdfs_dn_data_port }}"
• hive.server2.webui.port: "{{ hive_hiveserver2_webui_port }}"

The configured port are:

• hdfs_nn_rpc_port : 8020
• hdfs_dn_data_port : 9866
• hive_hiveserver2_webui_port : 0