datacommonsorg / website

Code for the Data Commons website
https://datacommons.org
Apache License 2.0
24 stars 88 forks source link

[CDC Autopush] Reorganize build, deploy, and test scripts #4499

Closed hqpho closed 4 months ago

hqpho commented 4 months ago

For ease of debugging workflow failures, I want to try having each script be its own build step. The new build config YAML will be:

steps:
  - id: clone-website-repo
    name: gcr.io/cloud-builders/git
    entrypoint: bash
    args:
      - -c
      - |
        set -e
        mkdir src
        git clone https://github.com/datacommonsorg/website.git src
        cd src
    waitFor: ['-']

  - id: build-and-tag-latest
    name: gcr.io/cloud-builders/docker
    entrypoint: bash
    args:
      - -c
      - |
        set -e
        ./scripts/build_custom_dc_and_tag_latest.sh
    waitFor: ['clone-website-repo']

  - id: deploy-latest-to-autopush
    name: gcr.io/cloud-builders/gcloud
    entrypoint: bash
    args:
      - -c
      - |
        set -e
        ./scripts/deploy_custom_dc_latest_to_autopush.sh
    waitFor: ['build-and-tag-latest']

  - id: run-tests
    name: python:3.11.3
    entrypoint: bash
    args:
      - -c
      - |
        ./run_test.sh --setup_python
        ./scripts/run_cdc_tests.sh
    waitFor: ['deploy-latest-to-autopush']

options:
  machineType: 'E2_HIGHCPU_32'
hqpho commented 4 months ago

Nice!

From the PR description:

 - id: run-tests
    name: python:3.11.3
    entrypoint: bash
    args:
      - -c
      - |
        ./run_test.sh --setup_python
        ./scripts/run_cdc_tests.sh
    waitFor: ['deploy-latest-to-autopush']

Looks like run_cdc_tests.sh is now setting up python itself so you don't need to do that in the build step.

Oh right! Ok I'll take out the setup python line and add a set -e line. Thanks for the quick review!