Hardcoded default cluster domain (svc.cluster.local)

[sammich]

Describe the bug

We use a non-default DNS domain name (i.e. not svc.cluster.local, our domains are in resolv.conf) and there are a couple of issues we're seeing:

1. The templated Hazelcast service hostname hardcodes the default domain pega-environment-config.yaml#L178. Locally changed to remove the svc.cluster.local still resulted in DNS lookups to that domain. Adding our domain to the end worked.
2. The Hazelcast pod is also doing lookups to the original service hostname (before change above, with svc.cluster.local) but we cannot find where it's getting this hostname. However the pod appears healthy, if polluting the logs.

To Reproduce

• a k8s cluster with a non-default DNS domain name
• build image with the necessary resolv.conf

Expected behavior

1. No lookup errors. Service resolves without assuming a FQDN with default k8s domain name
2. No DNS lookup errors in the logs.

Chart version 3.24.1

No customisations beyond the change mentioned in item 1 above. Otherwise, only values changes.

Server (if applicable, please complete the following information):

• Environment: Open Source Kubernetes 1.24