Google Cloud Platform Integration Feedback

[mcasperson]

Read about the new Google Cloud Platform integrations here.

We would love to hear your feedback! All feedback is welcome, and we would love to know:

• What deployments or operations tasks do you perform to GCP today?
• What are the pain points when deploying to or managing GCP?
• Do you use platforms like Google Kubernetes Engine, Google App Engine, Google Cloud Functions, Google Cloud Run, or others?
• Do the new features work for you? If not, what suggestions do you have that could improve them?

[StephenShamakian]

@mcasperson We haven't upgraded to v2021.2.x yet. But are planning on doing so Oct. 10th.

Here is some feedback we have on the GCP integration we have done pre-v2021.2.x:

- What deployments or operations tasks do you perform to GCP today? So far it is limited to dropping code to a GCP bucket.

- What are the pain points when deploying to or managing GCP? Latency Different teams using different code layouts for the same type of project (composer) Hosting the credentials on the server - Should be resolved with the version we are moving to.

- Do you use platforms like Google Kubernetes Engine, Google App Engine, Google Cloud Functions, Google Cloud Run, or others? So far: Composer Future: confluent kafka, Infoworks, Big Data

- Do the new features work for you? If not, what suggestions do you have that could improve them? Unknown

Regarding Cloud/Non-Cloud Credential Storage in Octopus Deploy: I also have a thought around storing cloud accounts in the Octopus Accounts section (not just GCP). It would be great if we could configure a Azure, AWS, GCP Octopus account (In the Infrastructure Tab) to directly connect to a third party secrets management tool like BeyondTrust/HashiCorp Vault. From a DevOps Engineering perspective, I push for storing Cloud accounts in the Octopus account section (easy variable scoping options in deploy projects). But we also have security mandates to store credentials in Enterprise approved secrets management products like BeyondTrust so it can auto rotate passwords/secrets. Right now we have to build an external automation script/runbook that keeps the Octopus Account in-sync with a rotated Cloud Account in Octopus.

On this topic, also being able to reference other style of account variables (User/Password/Token) in deploy variables would be helpful as again we need a place where we can store secrets that don't get snapshotted in a release as again we rely on third party secrets management tools. But the use of those tools need a "credential zero" of sorts too so being able to store those credentials in a central spot that doesn't get snapshotted makes it really easy to change the value of them.

[pio-janik]

@mcasperson We upgraded our Octopus Deploy to new version and we are very happy with the new features introduced in 2021.2.

Most our deployments are focused on using Google Kubernetes Engine and maintaining infrastructure by Terraform, but probably in feature We want to try different products: App Engine, Cloud Run and Cloud Functions(use mainly to track infrastructure events). Also We use Cloud SQL in several projects. The new feature grant us ability to store credentials in one place and audit them on-demand - old solution with store JSON credentials in VariableSet was quite painful. It would be great to see dedicated deployment step, that will allow to upload and download files from Google Cloud Storage.

[pio-janik]

@mcasperson Also if We talking about elements that are painful:

• Octopus could be better integrated with Google Cloud KMS and Secret Manager. KMS can be useful for easy rotate keys for our applications and infrastructure from one place.
• Octopus could auto install Terraform on machine if not present - with specific version. This can be make similarly like in step with Helm Upgrade, where Octopus send version from Octopus built-in feed.

[mcasperson]

Thanks for the feedback @pio-janik and @StephenShamakian .

Integration with external secret management tools is definitely something we hear a lot from customers. We do have some step templates to help, but unfortunately they don't address the use case mentioned here where targets store their credentials externally. This is a good scenario to support with any new feature that integrates with external secret managers.

so being able to store those credentials in a central spot that doesn't get snapshotted makes it really easy to change the value of them

There has been some discussion around being able to toggle whether variables are included in a snapshot or not. I agree it generally doesn't make sense to snapshot variables like passwords as they could change after release creation. There is no firm plan today for how or when this might be implemented, but this is good feedback.

Octopus could auto install Terraform on machine if not present

We did originally distribute terraform executables as part of a deployment, but found this was an unmanageable solution. Trying to keep on top of all the terraform versions for each operating system was not sustainable.

However, this functionality is something you can achieve with execution containers for workers. This performs a deployment inside a Docker image of your choice, and so you can install your preferred version of terraform in a Docker image and access it during a deployment.

One limitation of execution containers for workers is that the Docker image can't be based on a Linux distribution that uses Musl, like Alpine. Unfortunately the terraform docker images are based on Alpine and so can't be used. But you could create your own images with terraform. The docs here provide some more information on creating custom worker images.