use affected command with vercel's ignore-build-step

[cyrus-za]

Vercel has a ignore build step option which takes in any shell command (can run nodejs if needed) and depending on the exit code it will either build the project or skip it.

Is there a way to do something like nx affected:list which then lists the affected apps rather than opening up a dep-graph?

Then we can filter it with grep and see if a given app is affected and whether to build it on vercel.

[cyrus-za]

@vsavkin I've manage to find that theres a nx affected:apps command which can be used with --plain which gives me the string I need to check if a given app is affected, with some minor bash scripting.

The problem I am running into is that vercel doesn't clone the full repo. they do a depth=2 and when running git branch it thinks we're on master which we are not, so the default of --base=master does not work and nx affected:apps is empty.

@JamesHenry seems to have a lot of knowledge on how vercel works behind the scenes, so I am hoping we can find a solution. See this discussion for more info: https://github.com/vercel/vercel/issues/3166

[amanintech]

You might be talking about this ?

• https://nx.dev/latest/react/guides/deploy-nextjs-to-vercel#skipping-build-if-the-application-is-not-affected

[cyrus-za]

Yes I was, but that docs did not exist in May. In fact, I sync'd with the nx team to help create those docs after I found a solution

[SunStupic]

I think I found a better solution after seen @cyrus-za's solution. We can make affected:apps and affected:libs actually work with --base=origin/master by add the origin.

# Add origin for git repositry
git remote add origin $GIT_CREDENTIALS

# Fetch remote master branch
git fetch origin master 

# Install @nrwl/workspace in order to run the affected command
npm install --no-package-lock --no-save @nrwl/workspace typescript --prefer-offline

# Run the affected command, comparing latest commit to origin/master
npx nx affected:libs --base=origin/master | grep $VERCEL_PROJECT_NAME -q

# Store result of the previous command (grep)
IS_AFFECTED=$?

if [ $IS_AFFECTED -eq 1 ]; then
  echo "🛑 - Build cancelled"
  exit 0
elif [ $IS_AFFECTED -eq 0 ]; then
  echo "✅ - Build can proceed"
  exit 1
fi

[amanintech]

Is there any way to compare the branch we merging into ?

[srosato]

I came up with something a bit more advanced in order to support the following features when deploying an Nx monorepo with Vercel, so I thought I would share it here:

• Only build affected app based on an APP env var. I create one Vercel project per app I need to deploy.
• Skip the build when the target APP has no affected changes compared to the last successful build
• Only run qa (lint, tests, prettier) on affected files from that app, compared to the last successful build as affected-base

A few things to note:

• It involves using the Vercel API to get the last successful build (Vercel does not currently have an env var for that) so you need to add your token as an env var to each project
• It involves using git credentials as an env var for example https://your-user:your-passwd@gitlab.com/group/repo.git
• It involves using jq, since Vercel does not include it, I use node-jq
• I put the bash scripts within a folder called pipelines/

vercel-build-ignore.sh:

# This script determines whether the build needs to be performed or not for a given app/lib

# Dependent on
#   - APP env var. Name of the app to check. Change this to your application name!
#   - APP_TYPE env var. Choice of apps | libs
#   - GIT_CREDENTIALS in the form of https://your-user:your-passwd@gitlab.com/group/repo.git

git remote add origin $GIT_CREDENTIALS
git fetch --unshallow
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin

# We are checking out to local branches here in order for our custom yarn scripts to still use master and HEAD when using affected commands
git reset --hard origin/master
git checkout $VERCEL_GIT_COMMIT_REF

# Determine version of Nx installed
NX_VERSION=$(node -e "console.log(require('./package.json').devDependencies['@nrwl/workspace'])")
TS_VERSION=$(node -e "console.log(require('./package.json').devDependencies['typescript'])")

# Install a few requirements in order to run the affected command
npm install @nrwl/workspace@$NX_VERSION typescript@$TS_VERSION node-jq --no-package-lock --no-save --prefer-offline

AFFECTED_BASE=$(bash pipelines/determine-affected-base.sh)

npx nx affected:$APP_TYPE --plain --base=$AFFECTED_BASE | grep $APP -q

IS_AFFECTED=$?

if [ $IS_AFFECTED -eq 1 ]; then
  echo "🛑 - Build cancelled, the $APP app was not affected by the changes."
  exit 0
elif [ $IS_AFFECTED -eq 0 ]; then
  echo "✅ - Build can proceed"
  exit 1
fi

build.sh. I just override the build execution command to ./pipelines/build.sh:

# This script will QA and export only the given app on Vercel.
# Dependent on:
#   - APP env var. Name of the app to check. Change this to your application name!
#   - VERCEL_TOKEN. For making API calls to vercel and get deployment info. Not provided automatically.
#   - VERCEL_PROJECT_ID. Provided by Vercel automatically.
#   - VERCEL_ENV. Provided by Vercel automatically.
#   - VERCEL_GIT_COMMIT_REF. Provided by Vercel automatically.
# If you need to test this locally, provide those env vars.

PATH=node_modules/node-jq/bin:$PATH

AFFECTED_BASE=$(bash pipelines/determine-affected-base.sh)

rm -f pipelines/affected-deps pipelines/${APP}-deps dep-graph.json

# When we run nx affected, it gives us all components that were affected against the base branch.
# However, we want to only run QA on dependencies of the mentioned APP and therefore we use the power
# of jq with the nx depth graph to find out which dependencies to QA before exporting our APP

determine_app_dependencies() {
  yarn nx dep-graph --file=pipelines/dep-graph.json
  jq -r "[.graph.dependencies.${APP}[].target] | unique + [\"${APP}\"] | .[]" pipelines/dep-graph.json | sort > pipelines/${APP}-deps
}

determine_all_affected_deps_from_affected_base() {
  local affected_apps=$(echo $(yarn --silent nx affected:apps --base=$AFFECTED_BASE --plain) $(yarn --silent nx affected:libs --base=$AFFECTED_BASE --plain))
  echo $affected_apps | tr ' ' '\n' > pipelines/affected-deps
}

get_deps_to_exclude_from_build() {
  echo $(comm -13 pipelines/${APP}-deps pipelines/affected-deps)
  # e2e type apps are not included by nx affected:apps or affected:libs for some reason, so we add them here
  jq -r ".graph.nodes[] | select (.name != \"${APP}-e2e\") | select( .type == \"e2e\") | .name" pipelines/dep-graph.json
}

print_deps_result() {
  echo "--- ${APP} deps"
  cat pipelines/${APP}-deps
  echo "--- Affected deps"
  cat pipelines/affected-deps
  echo "--- To exclude"
  echo $(get_deps_to_exclude_from_build) | tr ' ' '\n'
}

perform_qa() {
  local excluded_deps=$(echo $(get_deps_to_exclude_from_build) | tr ' ' ',')
  yarn nx affected:lint --base=$AFFECTED_BASE --exclude=$excluded_deps
  yarn nx format:check --base=$AFFECTED_BASE --exclude=$excluded_deps
  yarn nx affected:test --base=$AFFECTED_BASE --exclude=$excluded_deps
}

export_app() {
  yarn nx run $APP:export
}

determine_app_dependencies
determine_all_affected_deps_from_affected_base
print_deps_result
perform_qa
export_app

determine-affected-base.sh:

PATH=node_modules/node-jq/bin:$PATH

# use last successful vercel build, if available
AFFECTED_BASE=$(curl -s --request GET \
  --url "https://api.vercel.com/v6/deployments?teamId=${VERCEL_ORG_ID}&projectId=${VERCEL_PROJECT_ID}&state=READY&limit=50" \
  --header "Authorization: Bearer ${VERCEL_TOKEN}" | jq -r "[.deployments[] | select(.meta.gitlabCommitRef == \"${VERCEL_GIT_COMMIT_REF}\")] | .[0].meta.gitlabCommitSha")

if [[ -z "$AFFECTED_BASE" ]]; then
  if [[ "$VERCEL_ENV" == "production" ]] ; then
    AFFECTED_BASE="master~1"
  else
    AFFECTED_BASE=master
  fi
fi

echo $AFFECTED_BASE

[github-actions[bot]]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.