gcloud-cli install taking 50+ seconds every build

[hawksight]

Orb version

gcp-gcr: circleci/gcp-gcr@0.6.1

Although I have slightly modified the job from this orb in order to load the workspace for docker build. So I based my job on build-and-push-image and has the following steps:

    steps:
      - checkout
      - gcp-gcr/gcr-auth:
          google-project-id: <<parameters.google-project-id>>
          google-compute-zone: <<parameters.google-compute-zone>>
          gcloud-service-key: <<parameters.gcloud-service-key>>
      - attach_workspace:
          at: /tmp/workspace
      - gcp-gcr/build-image:
          registry-url: <<parameters.registry-url>>
          google-project-id: <<parameters.google-project-id>>
          image: <<parameters.image>>
          tag: << parameters.tag >>
          extra_build_args: <<parameters.docker-build-args>>
      - gcp-gcr/push-image:
          registry-url: <<parameters.registry-url>>
          google-project-id: <<parameters.google-project-id>>
          image: <<parameters.image>>
          tag: <<parameters.tag>>

What happened

So we use this rebranded job in all our repos but find consistently across them all that using the machine executor, it takes 50+ seconds to run the installation of gcloud. This is caused by the gcp-gcr/gcr-auth step which seems to invoke install from the gcp-cli orb.

I get the reason here is that I'm using a machine executor, but I can't seem to find any from circle that includes the gcloud-cli already. That would cut out 50+ seconds from our CI.

For reference below is an example where its actually quicker to build the docker image than it is to have gcloud installed.

I haven't tried remote docker as I would probably need to mount the workspace to the remote docker for the build to work, but would doing that and using a docker executor be one way of having gcloud cli pre-installed?

Expected behavior

Ideally I want to not install gcloud and save 50+ seconds of time.

But the only way I can see that happening is if there are some circleci machines or docker images with gcloud already installed. Is that the case with the default executor for this module?

I don't think I can use a docker executor as the build-image command would then need to point to a remote docker instance to work?

Is this unique to us, or do other people see this step taking a long time?

[benjlevesque]

I'm having the same issue, and I solved it using a custom image, inspired from CircleCI-Public

FROM circleci/python:3.7

ARG GCLOUD_VERSION=268.0.0

# Downloading gcloud package
RUN cd ~

# Downloading gcloud package
RUN curl https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${GCLOUD_VERSION}-linux-x86_64.tar.gz > /tmp/google-cloud-sdk.tar.gz

# Installing the package
RUN sudo mkdir -p /usr/local/gcloud 
RUN sudo tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz
RUN sudo /usr/local/gcloud/google-cloud-sdk/install.sh

# Adding the package path to local
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin

This works well for the orbs' commands, but I believe to use the orbs' job, but it's not suitable for the orb's jobs, as far as I know (you can't specify an executor or docker image when using the job direclty in the workflow, can you?)

I feel the orbs should offer a way to execute the job with a docker image that already has the tools installed...

[KyleTryon]

Hello all, as described in your initial post, the orb will only install the GCP CLI if it is not found within the execution environment. Installing is an option but if the image provided to the job has the GCP CLI installed that will be the fastest solution.

While CircleCI does not yet supply any docker images with the GCP CLI installed, we do intend to release a suite of deployment images in the future.

Currently, it would be recommended to build a custom docker image using our next-generation base image as the base for your docker image. Simply install the GCP CLI and build and that should shave quite a bit of time assuming you hit the cache for the base image (which is growing daily). https://circleci.com/docs/2.0/circleci-images/#circleci-base-image

@benjlevesque provided a great example, the new base image is newer than this post.

For any suggestions to the platform directly (rather than an orb) please feel free to open a new idea or vote for an existing one on our ideas portal: https://ideas.circleci.com/ideas

Thank you.

[benjlevesque]

Hey @KyleTryon I've sent a PR to fix the part I mentioned regarding jobs and executors.

This enables to have a very short and use the build-and-push-image job, while enjoying the optimizations of a custom image.