search

seqeralabs / cx-field-tools-installer

Unofficial Terraform solution to help clients install Seqera Platform
Apache License 2.0
2 stars 1 forks source link

[ URGENT ] Assess risk of regreSSHion #104

Open gwright99 opened 2 days ago

gwright99 commented 2 days ago

This solution relies upon OpenSSH to run ssh and scp commands to push locally-generated files to the Tower VM. The recent revelation by Qualsys re: an OpenSSH vulnerability requires a review of this dependency to ensure client sites are not negatively impacted.

gwright99 commented 2 days ago

There are two methods we get OpenSSH onto the Tower host VM:

  1. AMI-specification during the EC2 resource definition
  2. Launch template init script

The init script does not install openssh, so this means we are leveraging the active package that comes natively with the AMI.

Mitigation Opportunities / Considerations:

  1. Rely on native upstream patching by Amazon. This relies on AWS releasing a new AMI and client sites running terraform apply again to pull a new AMI data source. Clients running with a container DB will lose their DB as part of the AMI upgrade process.

  2. Restrict SSH ingress CIDRs. This feature is already exposed by the solution. The default value is quite loose, but can be easily restricted to a tighter set of CIDRs. Furthermore, clients using an Instance Connect Endpoint implementation have additional defenses provided by AWS IAM permissions.

  3. Patch existing OpenSSH on server. This could be done by modifying the supplied AWS Launch Template or (potentially) running a command on the existing VM. The Launch Template solution will require the recreation of the VM, putting container DB implementations at risk. The in-place command (? sudo update openssh-server ?) could be a good short-term fix but I do not yet know the command necessary / if we have access to a patched version.

gwright99 commented 2 days ago

How to check the version of the OpenSSH server running:

  1. ssh onto target VM.
  2. Run ssh -V

Example:

$ ssh -V
OpenSSH_8.7p1, OpenSSL 3.0.8 7 Feb 2023
arnaualcazar commented 1 day ago

For new deployments, we will get updates since you are not pinning a specific version: https://github.com/seqeralabs/cx-field-tools-installer/blob/49646cf5b020a0d4a9816cb2238da8858cf0fd7a/006_ec2.tf#L11C13-L11C28 When it is already deployed, option 3 should be used. Amazon recommends how to do that: https://docs.aws.amazon.com/linux/al2023/ug/updating.html. It can also be managed from the SSM side.

gwright99 commented 15 hours ago

For new deployments, we will get updates since you are not pinning a specific version: https://github.com/seqeralabs/cx-field-tools-installer/blob/49646cf5b020a0d4a9816cb2238da8858cf0fd7a/006_ec2.tf#L11C13-L11C28 When it is already deployed, option 3 should be used. Amazon recommends how to do that: https://docs.aws.amazon.com/linux/al2023/ug/updating.html. It can also be managed from the SSM side.

Thanks for the comment @arnaualcazar, but it doesn't quite work that way. You are right that I will go out and find the latest AMI every time, but I give the users a control flag to indicate whether they want to apply the new AMI or not: https://github.com/seqeralabs/cx-field-tools-installer/blob/49646cf5b020a0d4a9816cb2238da8858cf0fd7a/006_ec2.tf#L50

This has been done for two reasons:

  1. Some installations run with a container db and changing the AMI will cause this to be destroyed.
  2. Some installations have to fill out paperwork/get permission to upgrade their hardware.

I suspect that a majority of folks will choose to let the AMI update automatically, but can't assume it's a given. I'll have a look at the patch-in-place option link you shared (ty!) to familiarize myself with the process.