Google Cloud Run Deployment

[j]

Is GCR supported with Backstage?

PS. Backstage looks awesome.

[stefanalund]

@j Backstage can make requests towards Google's APIs (see example in this plugin) -- making it possible to build a plugin for pretty much any Google Cloud Platform product/feature.

This looks like the right place to start: https://cloud.google.com/run/docs/reference/rest

[paolomainardi]

I would like to contribute on this, can I claim this issue ?

[benjdlambert]

@paolomainardi assigned you!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[OrkoHunter]

Re-opening since plugin suggestion issues are now exempt from stalebot.

[Jake-Mok-Nelson]

Is there a reason why this wouldn't work on Cloud Run?

[cc4i]

Any update for this issue?

[Jake-Mok-Nelson]

I looked into this a little and it seems like there's some long running polling services that run in the backend and the plugins seem to support that which means it can't really be stateless, starting only with HTTP calls. Would run well on AppEngine though but it's more expensive.

[mharrvic]

Hello guys, I made a step-by-step instruction guide on how to deploy backstage with Cloud Run via Terraform Notes: https://mharrvic.com/notes/backstage-deploy-with-cloudrun-via-terraform Repo: https://github.com/mharrvic/backstage-cloudrun-terraform Feel free to raise any issues you might encounter. Hope this helps!

[miguelpuiggarcia]

Hello @mharrvic, thanks for your cloud run tutorial!!, I got all working and deployed now, What is the configuration you use to import catalog entities from github? Mine seem not to be working in cloud run properly when making changes in GH and clicking refresh... All works locally though. This is my config:

catalog:
  providers:
    github:
      nandosUk:
        organization: 'NandosUK' # string
        catalogPath: '/catalog-info.yaml' # string
        filters:
          branch: '(main|master)' # regex to match main or master branches
          repository: '.*' # Regex to match any repository
        schedule: # optional; same options as in TaskScheduleDefinition
          # supports cron, ISO duration, "human duration" as used in code
          frequency: { minutes: 10 }
          # supports ISO duration, "human duration" as used in code
          timeout: { minutes: 15 }

[miguelpuiggarcia]

Got it all working with Cloud run. A Bit complex deployment: This is the full cloudbuild.yaml deployment:

steps:
  # Uncommenting this will take too long to build, better to run ./runProd.sh before committing
  # - name: 'node:16'
  #   id: 'Backstage backend'
  #   entrypoint: 'bash'
  #   args:
  #     - '-c'
  #     - |
  #       echo "Building Backstage backend"
  #       yarn install --frozen-lockfile
  #       yarn cache clean --all
  #       yarn tsc
  #       yarn build:backend

  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    id: 'Get initial secret variables needed'
    entrypoint: bash
    args:
      - '-c'
      - |
        echo "Retrieving GITHUB_APP_CREDENTIALS secret..."
        gcloud secrets versions access latest --secret=GITHUB_APP_CREDENTIALS > backstage-app-credentials-github.yaml

        export GOOGLE_APPLICATION_CREDENTIALS=$(gcloud secrets versions access latest --secret=BUCKET_CREDENTIALS_BACKSTAGE)
        export LOG_LEVEL=debug

  - name: 'gcr.io/cloud-builders/docker'
    args: ['pull', 'docker/dockerfile:experimental']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['pull', 'docker/dockerfile:1.0-experimental']
  - name: 'gcr.io/cloud-builders/docker'
    args:
      [
        'build',
        '.',
        '-f',
        'packages/backend/Dockerfile',
        '-t',
        '${_DOCKER_ARTIFACT_REGISTRY}/${_SERVICE_NAME}:${_BUILD_ENV}',
        '-t',
        '${_DOCKER_ARTIFACT_REGISTRY}/${_SERVICE_NAME}:$COMMIT_SHA',
      ]
    env:
      - 'DOCKER_BUILDKIT=1'

  - name: 'gcr.io/cloud-builders/docker'
    id: 'Push container image'
    args:
      ['push', '${_DOCKER_ARTIFACT_REGISTRY}/${_SERVICE_NAME}:${_BUILD_ENV}']

  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    id: 'Deploy Cloud Run Image'
    entrypoint: bash
    args:
      - '-c'
      - |

        echo "Retrieving GITHUB_APP_CREDENTIALS secret..."
        gcloud secrets versions access latest --secret=GITHUB_APP_CREDENTIALS > backstage-app-credentials-github.yaml
        export GOOGLE_APPLICATION_CREDENTIALS=$(gcloud secrets versions access latest --secret=BUCKET_CREDENTIALS_BACKSTAGE)

        export POSTGRES_PORT="5432"
        export POSTGRES_USER="backstage"
        export AUTH_OKTA_CLIENT_ID="xxxx"
        export AUTH_OKTA_AUTH_SERVER="xxxx"
        export GITHUB_APP_CREDENTIALS=$(cat backstage-app-credentials-github.yaml)
        export LOG_LEVEL=debug

        if [ "$_BUILD_ENV" == "dev" ] || [ "$_BUILD_ENV" == "preview" ]; then
          export BACKSTAGE_BASE_URL=https://preview-backstage.xxx.dev
          export POSTGRES_HOST="/cloudsql/$PROJECT_ID:europe-west2:backstage-d-preview"

        elif [ "$_BUILD_ENV" == "preprod" ]; then
          export BACKSTAGE_BASE_URL=https://preprod-backstage.xxx.dev
          export POSTGRES_HOST="/cloudsql/$PROJECT_ID:europe-west2:backstage-d-preprod"

        elif [ "$_BUILD_ENV" == "prod" ]; then
          export BACKSTAGE_BASE_URL=https://backstage.xxx.dev
          export POSTGRES_HOST="/cloudsql/$PROJECT_ID:europe-west2:backie3"
        fi

        echo "Deploying Run Service..."
        gcloud run deploy $_SERVICE_NAME \
        --image=${_DOCKER_ARTIFACT_REGISTRY}/${_SERVICE_NAME}:$_BUILD_ENV \
        --region=$_LOCATION \
        --service-account=$_SERVICE_ACCOUNT \
        --update-secrets AUTH_OKTA_CLIENT_SECRET=AUTH_OKTA_CLIENT_SECRET:latest,\
        OKTA_TOKEN=OKTA_TOKEN:latest,\
        PAGERDUTY_TOKEN=PAGERDUTY_TOKEN:latest,\
        POSTGRES_PASSWORD=POSTGRES_PASSWORD_APP_BACKSTAGE:latest,\
        SENTRY_TOKEN=SENTRY_TOKEN:latest,\
        JIRA_TOKEN=JIRA_TOKEN:latest,\
        GITHUB_AUTH_CLIENT_ID=GITHUB_AUTH_CLIENT_ID:latest,\
        GITHUB_AUTH_CLIENT_SECRET=GITHUB_AUTH_CLIENT_SECRET:latest \
        --set-env-vars AUTH_OKTA_CLIENT_ID=$$AUTH_OKTA_CLIENT_ID \
        --set-env-vars POSTGRES_PORT=$$POSTGRES_PORT \
        --set-env-vars POSTGRES_USER=$$POSTGRES_USER \
        --set-env-vars POSTGRES_HOST=$$POSTGRES_HOST \
        --set-env-vars LOG_LEVEL=$$LOG_LEVEL \
        --set-env-vars BACKSTAGE_BASE_URL=$$BACKSTAGE_BASE_URL \
        --set-env-vars AUTH_OKTA_AUTH_SERVER=$$AUTH_OKTA_AUTH_SERVER \
        --timeout=1m

timeout: 3000s
images:
  - '${_DOCKER_ARTIFACT_REGISTRY}/${_SERVICE_NAME}'

I have a shell script to generate the bundle.tar.gz locally as it takes lots in cloudbuild:

#!/bin/sh
echo "Running docker for production"
source ./environment.sh
yarn install --frozen-lockfile
yarn cache clean --all
# tsc outputs type definitions to dist-types/ in the repo root, which are then consumed by the build
yarn tsc
# Build the backend, which bundles it all up into the packages/backend/dist folder.
yarn build:backend

environment.sh looks like this (without the values)

export POSTGRES_HOST=
export POSTGRES_PORT=""
export POSTGRES_USER=""
export POSTGRES_PASSWORD=""
export AUTH_OKTA_CLIENT_ID=""
export AUTH_OKTA_CLIENT_SECRET=""
export AUTH_OKTA_AUTH_SERVER=""
export OKTA_TOKEN=""
export PAGERDUTY_TOKEN=""
export SENTRY_TOKEN=""
export JIRA_TOKEN=""
export GITHUB_APP_CREDENTIALS="$(cat backstage-app-credentials-github.yaml)"
export GITHUB_AUTH_CLIENT_ID=""
export GITHUB_AUTH_CLIENT_SECRET=""
export BACKSTAGE_BASE_URL=http://localhost:3000

[benjdlambert]

@miguelpuiggarcia I wonder if you could create some docs under the deployments section so that other people can find information on how to deploy using Cloud Run and we could close this ticket? :pray:

[cathex-sam-holdaway]

@miguelpuiggarcia I wonder if you could create some docs under the deployments section so that other people can find information on how to deploy using Cloud Run and we could close this ticket? 🙏

Hi @benjdlambert. I'd happily contribute, I'm building out a simple Cloud Build pipeline - to deploy to Cloud Run - myself and hopefully can help save some pain (including my own). @miguelpuiggarcia I'll be in touch...

[miguelpuiggarcia]

@miguelpuiggarcia I wonder if you could create some docs under the deployments section so that other people can find information on how to deploy using Cloud Run and we could close this ticket? 🙏

Hi @benjdlambert. I'd happily contribute, I'm building out a simple Cloud Build pipeline - to deploy to Cloud Run - myself and hopefully can help save some pain (including my own). @miguelpuiggarcia I'll be in touch...

Thanks @cathex-sam-holdaway Yes its on my todo list as well to document the cloud run. If you can move the first stone would be great :)

[cathex-sam-holdaway]

@miguelpuiggarcia I wonder if you could create some docs under the deployments section so that other people can find information on how to deploy using Cloud Run and we could close this ticket? 🙏

Hi @benjdlambert. I'd happily contribute, I'm building out a simple Cloud Build pipeline - to deploy to Cloud Run - myself and hopefully can help save some pain (including my own). @miguelpuiggarcia I'll be in touch...

Thanks @cathex-sam-holdaway Yes its on my todo list as well to document the cloud run. If you can move the first stone would be great :)

Sent you an email 👍

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.