[Bug] self-hosted temporal with AWS RDS (Unable to connect to SQL database)

MayurDuduka commented 5 months ago

What are you trying to do?

Trying to deploy self-hosted Temporal on AWS EKS cluster with AWS Postgres RDS as persistence layer. I'm following this custom values file for reference. however, the setup-schema job is failing with the below-mentioned error.

2024-02-12T12:45:27.121Z    ERROR   Unable to connect to SQL database.  {"error": "pq: no pg_hba.conf entry for host \"x.x.x.x\", user \"postgres\", database \"temporal_visibility\", no encryption", "logging-call-at": "handler.go:73"}
I tried with auto-setup k8s job and admintools pod manually by running temporal-sql-tool commands. but error is same.
I'm able to connect with postgres using postgres-client in the kubernetes cluster as well as using pgadmin.

ip which is showing up in the error, that's not from RDS dns.
also tried updating the parameter group of rds for pg_hba.conf to rds.force_ssl=0

Describe the bug

Minimal Reproduction

get eks cluster version 1.28
get existing rds postgres 15.4
deploy temporal using helm from release 0.33.0 with this custom-values file as a reference.

Environment/Versions

Temporal release Version: 0.33.0
Temporal version: 1.22.4
EKS Kubernetes: 1.28

Additional context

luismacosta commented 5 months ago

Hi @MayurDuduka

I had the same issue The error shows "no encryption" The default rds.forcessl parameter is set to 1 since RDS PostgreSQL version 15 You need to run temporal-sql-tool with parameters --tls --tls-ca-file bundle.pem_ Regarding bundle.pem file, certificate bundles for specific aws regions can be found here Also, in values.yaml, enable tls sections:

tls:
  enabled: true

robholland commented 1 month ago

The schema jobs were not correctly configured to setup TLS. This has been fixed in https://github.com/temporalio/helm-charts/pull/411.

temporalio / helm-charts