Make release process less error-prone

[remcotolsma]

When releasing a plugin we have to check a few things:

• Version number in package.json.
• Version number in readme.txt.
• Version number in plugin.php.

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/package.json#L3

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/readme.txt#L8

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/pronamic-ideal.php#L7

In this repository we test this with https://phpunit.de/:

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/tests/src/VersionNumberTest.php#L37-L62

On other (add-on) plugins we don't always test this, this results sometimes in broken releases:

https://github.com/pronamic/wp-pronamic-pay-contact-form-7/releases/tag/3.1.2

How can we improve and simplify the release process?

• https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
• https://github.com/typicode/husky
• https://docs.npmjs.com/cli/v7/commands/npm-version

How do other WordPress plugins manage this?

• https://github.com/woocommerce/woocommerce
• https://github.com/Yoast/wordpress-seo
• https://github.com/WordPress/gutenberg

[rvdsteege]

In the interactive release via php src/repositories.php release the package.json files are updated with the selected version number for the extension and gateway repositories. As the exact new version number is known at that point, it would be nice to somehow also update the readme.txt (command ??? > readme.txt?) and plugin.php.

[remcotolsma]

It would be nice if we have a solution that is also easy to use for other plugins. I read that npm version can also update package.json.

Run this in a package directory to bump the version and write the new data back to package.json, package-lock.json, and, if present, npm-shrinkwrap.json.

https://docs.npmjs.com/cli/v7/commands/npm-version#description

Perhaps this command can replace code like:

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/src/repositories.php#L265-L266

We could also use npm pkg set version=1.0.0?

https://docs.npmjs.com/cli/v7/commands/npm-pkg

Maybe a GitHub workflow should be the starting point for a release?

https://github.com/WordPress/gutenberg/blob/cbaa18495869932e04eaa1cac82b86001053d352/.github/workflows/upload-release-to-plugin-repo.yml

I did notice that this workflow is also using sed to update the version number in the readme.txt file.

https://github.com/WordPress/gutenberg/blob/cbaa18495869932e04eaa1cac82b86001053d352/.github/workflows/upload-release-to-plugin-repo.yml#L156-L161

            - name: Update the plugin's stable version
              working-directory: ./trunk
              run: |
                  sed -i "s/Stable tag: ${STABLE_VERSION_REGEX}/Stable tag: ${VERSION}/g" ./readme.txt
                  svn commit -m "Releasing version $VERSION" \
                   --no-auth-cache --non-interactive  --username "$SVN_USERNAME" --password "$SVN_PASSWORD"

https://www.gnu.org/software/sed/manual/sed.html

https://github.com/WordPress/gutenberg/blob/d07ad8ba66a614ab2b73af315a47371efdcd4a61/.github/workflows/build-plugin-zip.yml#L120-L126

            - name: Update plugin version
              env:
                  VERSION: ${{ steps.get_version.outputs.new_version }}
              run: |
                  cat <<< $(jq --tab --arg version "${VERSION}" '.version = $version' package.json) > package.json
                  cat <<< $(jq --tab --arg version "${VERSION}" '.version = $version' package-lock.json) > package-lock.json
                  sed -i "s/${{ steps.get_version.outputs.old_version }}/${VERSION}/g" gutenberg.php

https://github.com/WordPress/gutenberg/commit/f07982a9c22dd84f3810870e22bd506fc525ffa9

• https://github.com/wp-cli/i18n-command/blob/85b8ea9610e37c6c49cccbf9ea40badcf1a297fd/src/MakePotCommand.php#L459-L536
• https://github.com/WordPress/gutenberg/tree/trunk/packages/scripts#plugin-zip
• https://www.npmjs.com/package/wp-inject-config
• https://developer.wordpress.org/reference/functions/get_plugin_data/

[rvdsteege]

Maybe a GitHub workflow should be the starting point for a release?

Sure, that would be really nice. But how would that be initiated, manually via web?

npm version seems well suited for the job, as it also allows for running additional pre and post version change scripts.

[remcotolsma]

4. Run the version script. These scripts have access to the new version in package.json (so they can incorporate it into file headers in generated files for example). Again, scripts should explicitly add generated files to the commit using git add.

Is it an idea that we develop 2 scripts like npm pkg set version=1.0.0 that can update information in readme.txt and plugin.php?

npm run pronamic-wp-plugin-readme set "Stable tag" "1.0.0"  

npm run pronamic-wp-plugin-file set "Version" "1.0.0"

• https://developer.wordpress.org/plugins/plugin-basics/header-requirements/#header-fields
• https://developer.wordpress.org/plugins/wordpress-org/how-your-readme-txt-works/

Could be part of https://github.com/pronamic/wp-deployer or maybe a WP-CLI package similar like how we use wp i18n make-pot in plugin development.

[remcotolsma]

It should also work with style.css: https://developer.wordpress.org/themes/basics/main-stylesheet-style-css/

• https://codex.wordpress.org/File_Header
• https://codex.wordpress.org/File_Header_API

pronamic-wp-file-header style.css set "Version" "1.0.0"
pronamic-wp-file-header readme.txt set "Stable tag" "1.0.0"
pronamic-wp-file-header plugin.php set "Version" "1.0.0"

https://www.npmjs.com/package/wp-file-header-metadata

[remcotolsma]

I did some work in https://github.com/pronamic/wp-file-header.

vendor/bin/pronamic-wp-file-header set Version $( npm pkg get version | xargs echo ) pronamic-pay-digiwallet.php

https://github.com/pronamic/wp-pronamic-pay-digiwallet/blob/3eb0e15f970b2a34ece08760c08513a6b1828ba3/package.json#L41

This would be nice:

• https://github.com/npm/cli/issues/5508
• https://github.com/npm/rfcs/issues/634

[remcotolsma]

Just discussed with @rvdsteege @pronamic HQ, a few things to consider:

Check Version Numbers

Before npm version we want to check if the version numbers in package.json, plugin.php, style.css and/or readme.txt match.

The https://github.com/pronamic/wp-file-header library can help to retrieve the values from plugin.php, style.css and/or readme.txt.

WP React Starter

Never adjust package.json's version and CHANGELOG.md manually!

• https://devowlio.gitbook.io/wp-react-starter/advanced/build-production-plugin
• https://lerna.js.org/docs/features/version-and-publish
• https://github.com/lerna/lerna/tree/main/commands/version

Pronamic Pay

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/tests/src/VersionNumberTest.php#L15-L63

Changelog

For each release we want to make sure that there is a changelog.

https://keepachangelog.com/en/1.0.0/

Can changelogs be automatically parsed?

It’s difficult, because people follow wildly different formats and file names.

Vandamme is a Ruby gem created by the Gemnasium team and which parses many (but not all) open source project changelogs.

Gutenberg

            - name: Build release notes draft
              if: ${{ needs.bump-version.outputs.new_version }}
              env:
                  VERSION: ${{ needs.bump-version.outputs.new_version }}
              run: |
                  IFS='.' read -r -a VERSION_ARRAY <<< "${VERSION}"
                  MILESTONE="Gutenberg ${VERSION_ARRAY[0]}.${VERSION_ARRAY[1]}"
                  npm run other:changelog -- --milestone="$MILESTONE" --unreleased > release-notes.txt
                  sed -ie '1,6d' release-notes.txt
                  if [[ ${{ needs.bump-version.outputs.new_version }} != *"rc"* ]]; then
                    # Include previous RCs' release notes, if any
                    CHANGELOG_REGEX="=\s[0-9]+\.[0-9]+\.[0-9]+(-rc\.[0-9]+)?\s="
                    RC_REGEX="=\s${VERSION}(-rc\.[0-9]+)?\s="
                    awk "/${RC_REGEX}/ {found=1;print;next} /${CHANGELOG_REGEX}/ {found=0} found" changelog.txt >> release-notes.txt
                  fi
            - name: Upload release notes artifact
              if: ${{ needs.bump-version.outputs.new_version }}
              uses: actions/upload-artifact@e448a9b857ee2131e752b06002bf0e093c65e571 # v2.2.2
              with:
                  name: release-notes
                  path: ./release-notes.txt

https://github.com/WordPress/gutenberg/blob/d07ad8ba66a614ab2b73af315a47371efdcd4a61/.github/workflows/build-plugin-zip.yml#L188-L209

WooCommerce

      - name: "Generate the changelog file"
        run: pnpm changelog --filter=woocommerce -- write --add-pr-num -n -vvv --use-version ${{ inputs.releaseVersion }}

https://github.com/woocommerce/woocommerce/blob/a1f5b71e272213006b81c51135e8d5cb47bae919/.github/workflows/release-changelog.yml#L43-L44

            - name: Update the Changelog to include the release notes
              run: |
                  # First, determine where to insert the new Changelog entry.
                  SERIES="${RELEASE_BRANCH#release/}"
                  SERIES_REGEX="=\s${SERIES}\.[0-9]+\s="
                  CUT_MARKS=$( grep -nP -m 1 "${SERIES_REGEX}" changelog.txt | cut -d: -f1 )
                  if [[ -z "${CUT_MARKS}" ]]; then
                    CHANGELOG_REGEX="=\s[0-9]+\.[0-9]+\.[0-9]+(-rc\.[0-9]+)?\s="
                    RC_REGEX="=\s${TAG#v}(-rc\.[0-9]+)?\s="
                    CUT_MARKS=$( awk "/${RC_REGEX}/ {print NR; next}; /${CHANGELOG_REGEX}/ {print NR; exit}" changelog.txt )
                  fi
                  BEFORE=$( echo "$CUT_MARKS" | head -n 1 )
                  AFTER=$( echo "$CUT_MARKS" | tail -n 1 )
                  # Okay, we have all we need to build the new Changelog.
                  head -n $(( "${BEFORE}" - 1 )) changelog.txt > new_changelog.txt
                  printf '= %s =\n\n' "${TAG#v}" >> new_changelog.txt
                  # Need to use a heredoc in order to preserve special characters.
                  cat <<- "EOF" > release_notes.txt
                  ${{ github.event.release.body }}
                  EOF
                  # Normalize empty lines: Trim them from beginning and end of file...
                  awk 'NF {p=1} p' <<< "$(< release_notes.txt)" >> new_changelog.txt
                  # ...then add two empty lines at the end.
                  printf '\n\n' >> new_changelog.txt
                  tail -n +"${AFTER}" changelog.txt >> new_changelog.txt
                  mv new_changelog.txt changelog.txt

https://github.com/WordPress/gutenberg/blob/0c807b34bf83a845599eb4d3da445ba8f288a1fc/.github/workflows/upload-release-to-plugin-repo.yml#L47-L72

https://github.com/woocommerce/woocommerce/blob/a1f5b71e272213006b81c51135e8d5cb47bae919/.github/workflows/scripts/release-changelog.php

Pronamic Pay

https://github.com/pronamic/wp-pronamic-pay/blob/afa120bd38569741402447333c0572aa98b38b4d/src/repositories.php#L268-L285

[remcotolsma]

I have made some progress today, next thing is the changelog, some resources / ideas:

• https://github.com/chelnak/gh-changelog
• https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes
• https://mokkapps.de/blog/how-to-automatically-generate-a-helpful-changelog-from-your-git-commit-messages/
• https://www.philipp-doblhofer.at/en/blog/automatic-changelog-and-versioning-with-git/
• https://github.com/KeNaCo/auto-changelog
• https://www.conventionalcommits.org/en/v1.0.0/
• https://github.com/cookpete/auto-changelog
• https://github.com/marcocesarato/php-conventional-changelog
• https://github.com/pyrech/composer-changelogs