Configure load balancers to only accept SSL connections - Githubissues

unity-sds / unity-sps

The Unity SDS Processing Service facilitates large-scale data processing for scientific workflows.

Apache License 2.0

2 stars 2 forks source link

Configure load balancers to only accept SSL connections #175

Closed nikki-t closed 2 months ago

nikki-t commented 2 months ago

Purpose

Configure the SPS load balancers (for Airflow and OGC API) to only accept SSL connections.
Modifications serve HTTPS traffic on port 4443.

New endpoint URLS:

Airflow UI: https://k8s-airflow-airflowi-xxx-xxx.xxx.com:4443/home
Airflow API: https://k8s-airflow-airflowi-xxx-xxx.xxx.com:4443/api/v1/
OGC API: https://k8s-airflow-ogcproce-xxx-xxx.xxx.com:4443
OGC UI: https://k8s-airflow-ogcproce-xxx-xxx.xxx.com:4443/redoc

Note: The SSL cert is for a different domain than individual deployments so you will have to manually accept the certificate when accessing any of the endpoint URLs in a web browser.

Proposed Changes

[ADD] Configure SPS load balancers to only accept SSL connections.

Issues

128 - Enable SSL on SPS load balancers

Testing

Deployed to unity-venue-dev and reviewed Application Load Balancer definition:

Verified load balancer security group includes TCP:4443
Verified Event Listener and rule for HTTPS:4443
Verified target group
Verified updated SSM parameters

Tested endpoint URLs:

Airflow UI and OGC UI: Can access by accepting the risk for the SSL domain mismatch.
Airflow API: Listed available DAGs via https://xxx.com:4443/api/v1/dags
OGC API: Registered and deployed a process via this documentation
Also checked health endpoints.

Note: Needed to use the following Terraform commands to update existing Load Balancers (Airflow and OGC):

terraform apply -replace="module.unity-sps-airflow.kubernetes_ingress_v1.ogc_processes_api_ingress" -replace="module.unity-sps-airflow.kubernetes_ingress_v1.airflow_ingress" --var-file=tfvars/${TFVARS_FILENAME}

nikki-t commented 2 months ago

@LucaCinquini - I don't think have to use 4443 for both the Airflow and OGC endpoints. I believe we can use whatever port we would like. I will keep the ports at 5000 and 5001 and test to be sure.

I was following this documentation which suggest using 4443 but I think it's by convention.

nikki-t commented 2 months ago

The ports have been modified back to their original port numbers (5000 - Airflow and 5001 - OGC) and are defined with HTTPS and an SSL certificate so traffic is served over HTTPS.

I deployed these changes, tested, and was able to confirm everything works as expected.

nikki-t commented 2 months ago

I have updated the smoke tests and ran them both locally and via GitHub Actions, here: https://github.com/unity-sds/unity-sps/actions/runs/10146052219/job/28053215683.

I ran the test with URLs entered for the MCP Venue Dev environment but not for the others as I wasn't quite sure what they would be. Despite that the tests did succeed.