GitHub actions are an almost endless source of features that will allow you to trigger on any kind of event that happens on GitHub. It starts a workflow which is executed in a container injected with your source code.
[!NOTE]
☝️ Learning goals in this issue:
Understand the workflow yaml file; on, env, permissions, jobs, usesand run.
See the gh CLI in action
Use the GitHub REST API for additional stuff gh doesn't support (yet)
Log messages to the action execution using different log levels
Use the GitHub web interface to create new workflows from 100's of predefined templates
[!TIP]
This issue is independent from all other issues - you can skip this if you want to do some of the others first
GitHub actions are extensions which are effectively repos on GitHub. You can then use them from the workflow (.yml file).
Some actions are officially published but you and your team can just create your own actions and store them on GitHub and use then directly from there - no official release is required for it to be available.
The actions that trigger are defined in a workflow file - and since the runtime environment for the execution is a linux based container you can literally runanything that you can script - either embedded in the workflow file or in a separate script you call from the the workflow.
The runtime environment in which the action is triggered is default pre-authenticated (you can turn it off) so it can access the GitHub eco-system with the same credentials as yourself.
GitHub has a full fledged API that allows you to do access all GitHub features. But it's tedious and verbose to use an API like that from a shell script. In January 2020 - to make life easier for developers automating workflows - GitHub released the first version of the GitHub CLI gh. Although it doesn't (yet) cover all features it's amazingly comprehensive. It gives you command line access to 'all' GitHub features - from the terminal. Which means that your actions can do anything.
Let's examine the workflow that was triggered on repo-creation - when you pushed "Use this template".
Here's the context - the user story - so to speak:
Problem
Solution
Value
When I do tutorials and training materials in GitHub I want to present the material and exercises as issues. But both forks and templates on GitHub ignores the GitHub issues from the source repo — they are simply not copied over in the new repo.
Use GitHub Actions to start a workflow that triggers on the creation of a repo based on using another repo as a template. In that workflow use the GitHub CLI gh to reach back into the source repo, read the issues there and recreate them in the copy.
Issues from the source are now conceptually part of the template. From the perspective of didactics it has the advantage that meta data is kept out from cluttering the git filesystem. Rather they are stored in the same place at developers would naturally have that kind of meta data. Learners get to familiarize themselves with GitHub issues too and they'll have a natural place to record notes on their individual learning process.
🏋️♀️ Exercise
[ ] 👉 Go to the Actions panel and open the workflow that just ran on "Initial commit" 👈
[ ] 👉 Open the workflow file that ran this particular workflow" 👈
In the left panel you'll find a read-only reference to the workflow file, in the version as it looked in the actual commit that triggered the workflow.
[ ] 👉 Find the original workflow file in the git repo at /.github/worflows/template.yml and open that in the web editor 👈
You'll see the same file, but in the editable version from main branch. Since you are now in a genuine VS Code IDE you'll have the IntelliSense support you expect.
As an example; if you hover on: you'll get instant access to a tool-tip box on this particular keyword. Apparently VS Code has built in support for GitHub Actions Workflow files.
🏋️♀️ Exercise
[ ] 👉 Hover over the keywords in the workflow file and see how much you'll learn about GitHub Actions 👈
A rundown on what goes on in this particular workflow:
In the rest of this particular issue I'll not throw exercises at you, but rater walk you through some of the finer details in this flow. If you are already familiar with the semantics of the workflow files you may skip it - and then again; It does take you through some special cases, that serves to clarify the point, that a lot can be done with small means, by standing on the shoulders of other's general abstractions: Fight software complexity with more software!
Workflow: /.github/worflows/template.yml
Fire unconditionally on any branch (Lines 1-6)
```yaml
name: Initial commit workflow
on:
push:
branches:
- '**'
```
`name` is optional, it serves as a heading in the workflow execution.
`on`- defines what should trigger this workflow. Since there is not an actual event for creating a new repo the hack is to use `'**'` this will make the trigger [fire on _any_ branch](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#patterns-to-match-file-paths).
Obviously firing unconditionally on _anything_ may cause problems in the long run so this approach requires special measures, as you'll see later
Authenticating git and the gh CLI(Lines 8-9)
```yaml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```
`env` is the section that defines environment variables. Setting `GITHUB_TOKEN` is the standard way of authenticating the GitHub CLI. GitHub actions support _secrets_ and GitHub automatically generate a token when starting a workflow, it's stored in the _secrets_ so this becomes that [standard way of using the automatic token authentications](https://docs.github.com/en/actions/security-guides/automatic-token-authentication)
Assigning permissions(Lines 11-15)
```yaml
permissions:
checks: write
actions: write
contents: write
issues: write
```
permissions is used to change the permission if you need anything other than the default permissions. For this particular run I had to add explicit write permissions to actions (as you'll see later I'll disable this flow as soon as it has run once) and issues so the workflow can create new issues and even contents to update the readme for the new repo coming off the template.
Avoid multiple concurrent runs (Lines 17-19)
```yaml
concurrency:
group: "cpissues"
cancel-in-progress: false
```
Due to the aggressive pattern `**` we could risk multiple jobs being triggered, dependent on enduser behaviour. But we can only allow one run at a time. and any new jobs can not be allowed to stop ongoing jobs. The default behaviour is to allow concurrency but it can easily be altered using the [concurrency features](https://docs.github.com/en/actions/using-jobs/using-concurrency).
Define the image to use (Lines 21-24)
```yaml
jobs:
cpissues:
name: "Initial commit only"
runs-on: ubuntu-latest
```
`jobs` is where the action happens - the section defines all the jobs that the workflow knows of. Each job has it's on tag (you define the name yourself) in this case I only have one job and I came up with the clever name `cpissues`.
`runs-on` specifies the runner and image that this workflow will run in. I only need to do small bits in the terminal, so I'm happy with the default _latest Ubuntu_ But even though `runs-on` is just a small clause it has a huge impact and by [defining your own runners and images](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idruns-on) you can open up a complete customized world to [_Platform Engineering_ and _Internal Development Platforms_](https://platformengineering.org/blog/what-is-platform-engineering) where you can configure the actions to support _anything as code_ - quite literally _"anything"!_ The finer details of the world that opens up underneath `runs-on` is beyond the scope of this tutorial. So go and explore yourself.
Checkout the code (Lines 25-28)
```yaml
if: ${{ contains(github.event.head_commit.message, 'Initial commit') }}
steps:
- name: "Checkout"
uses: actions/checkout@v3
```
`steps` defines the different steps executed in this workflow. Each step becomes an expandable/collapsable section in the job execution log you saw earlier, so defining smalle elegant steps is a means to keep a tidy and readable log of in your workflows.
The rest of the this job happens in the context of these `steps` but before the workflow continues an `if` statement is used to fix the inconvenient fact that there is not a dedicated event that fires on repository creation only the greedy `**`which fires on any branch". The steps are simply ignored, unless the commit message of the commit spells exactly `'Initial commit'`. Because this is exactly the commit message the GitHub _always_ uses when it creates new repos off of templates.
`uses` is referring to _actions._ Actions are called by their location on GitHub so consequently the GitHub Action `actions/checkout` resides on [https://github.com/actions/checkout](https://github.com/actions/checkout). And it uses the package release feature to offer a version `3`.
The [@actions](https://github.com/actions) organization on GitHub is controlled and maintained by GitHub self. So you know, that if an action is located here it's considered GitHub standard. Almost every workflow will call this particular `actions/checkout` action as the first. It checks out the repo in the runner, and makes sure, that the rest of the job execution has access to the content of the repo.
Cancel the workflow when it's done (Lines 30-38)
```yaml
- name: "Raise semaphore (cancel workflow) or quit"
run: |
if $(gh workflow disable template.yml); then
echo ::notice title=Workflow successfully disabled::This also works as a raised semaphore - parallel workflow runs will see it and cancel
else
echo "::warning title=Another job got here first::This room is crowded - I'm outta here!"
gh api -X POST /repos/{owner}/{repo}/actions/runs/${{ github.run_id }}/cancel
exit 1
fi
```
`run` is similar to `uses` but where `uses` calls a predefined action, `run` executes native shell commands. Starting the `run` clause with a pipe `|` followed by a new-line is a notation used to make the rest of the clause - a shell script.
The first thing that happens is that the workflow `template.yml` is disabled. This is done simply by using the `gh` cli. The command will raise an error if it doesn't succeed - like if it's already disabled.
The next line demonstrate how to [use simple `echo` statements to create messages in the workflow execution log](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-debug-message) four levels are supported: `debug`, `notice`, `warning` and `error`. Perhaps you noticed the output from this earlier, when you browsed the content on the `Actions` panel.
In the rare event that two (or more) job executions were queued the later ones will fail in disabling the workflow - since it's already disabled - and end the `else` clause. This is a precaution that deals with the occasional abnormality that sometimes GitHub fires the workflow twice on the same catch-all `'**'` branch pattern. When that is the case, I'd rather have _this_ workflow cancelled, than having the record of a failed workflow. However cancelling a running instance of a workflow is is not supported from the `gh` CLI. but it _its_ supported by a webhook on the GitHub REST API. And `gh` gives access to calling this API with the right GitHub Authentication:
```shell
gh api -X POST /repos/{owner}/{repo}/actions/runs/${{ github.run_id }}/cancel
```
Consequently this worflow now works as intended: It runs once - and only once, and only on the initial commit. And it doesn't take up any polling resources after, because it's effectively disabled.
Copy issues over, using a gh extension (Lines 40-43)
```yaml
- name: "Copying issues over from template repo"
run: |
gh extension install thetechcollective/gh-cpissues
gh cpissues thetechcollective/dx-intro --label template
```
Instead of having a lengthy script doing a lot of `gh`, `json`, `base64` and `jq` gymnastics to copy issues over - I put all of that nitty gritty stuff into a separate script and stored it as `gh` extension on [github.com/thetechcollective/gh-cpissues](https://github.com/thetechcollective/gh-cpissues).
[`gh` extensions](https://docs.github.com/en/github-cli/github-cli/using-github-cli-extensions) works very much like regular `git` extension: If the script is named `gh-` then `` is now a `gh` subcommand. And `gh` comes with the highly desirable added feature that it has a built in package manager feature for exactly these types of repos.
It's crazy easy to create your own [`gh` extension](https://docs.github.com/en/github-cli/github-cli/creating-github-cli-extensions) and it's a brilliant way to reuse the code snippets and features you create for you workflows. Personally I often even favour them over creating my own GitHub Actions, because the `gh` extensions can run in _any_ terminal with `gh` installed and are therefore independent from the GitHub Action eco-system.
This is the simple syntax of the `cpissues` extension:
``` shell
USAGE:
gh cpissues --label Set the new README.md (Lines 45-43)
```yaml
- name: "Set the README"
run: |
cp .github/template/README.md .
rm -rf .github/template/
git config --global user.email "actions.bot@github.com"
git config --global user.name "The GitHub Action Bot"
git add -A
git commit -m "Updated the README"
git push origin
```
This bit uses `run` to replace the `README.md` in the root with the one that's designed, to be shown in the copy. This needs to be committed and pushed, and in order to do that we need to verify to git who we are by `name` and `email`.
This creates a new commit - If you go and take a look, your repo already has two commits, but also notice that you only have one job execution - despite that the workflow subscribes to _any_ branch `'**'`: The disablement of the job worked as intended.
GitHub actions are an almost endless source of features that will allow you to trigger on any kind of event that happens on GitHub. It starts a workflow which is executed in a container injected with your source code.
GitHub actions are extensions which are effectively repos on GitHub. You can then
usethem from the workflow (.ymlfile).Some actions are officially published but you and your team can just create your own actions and store them on GitHub and
usethen directly from there - no official release is required for it to be available.The actions that trigger are defined in a workflow file - and since the runtime environment for the execution is a linux based container you can literally
runanything that you can script - either embedded in the workflow file or in a separate script you call from the the workflow.The runtime environment in which the action is triggered is default pre-authenticated (you can turn it off) so it can access the GitHub eco-system with the same credentials as yourself.
GitHub has a full fledged API that allows you to do access all GitHub features. But it's tedious and verbose to use an API like that from a shell script. In January 2020 - to make life easier for developers automating workflows - GitHub released the first version of the GitHub CLI
gh. Although it doesn't (yet) cover all features it's amazingly comprehensive. It gives you command line access to 'all' GitHub features - from the terminal. Which means that your actions can do anything.Let's examine the workflow that was triggered on repo-creation - when you pushed "Use this template".
Here's the context - the user story - so to speak:
forksandtemplateson GitHub ignores the GitHub issues from the source repo — they are simply not copied over in the new repo.template. In that workflow use the GitHub CLIghto reach back into the source repo, read the issues there and recreate them in the copy.template. From the perspective of didactics it has the advantage that meta data is kept out from cluttering the git filesystem. Rather they are stored in the same place at developers would naturally have that kind of meta data. Learners get to familiarize themselves with GitHub issues too and they'll have a natural place to record notes on their individual learning process.You'll see the same file, but in the editable version from
mainbranch. Since you are now in a genuine VS Code IDE you'll have the IntelliSense support you expect.As an example; if you hover
on:you'll get instant access to a tool-tip box on this particular keyword. Apparently VS Code has built in support for GitHub Actions Workflow files.A rundown on what goes on in this particular workflow:
In the rest of this particular issue I'll not throw exercises at you, but rater walk you through some of the finer details in this flow. If you are already familiar with the semantics of the workflow files you may skip it - and then again; It does take you through some special cases, that serves to clarify the point, that a lot can be done with small means, by standing on the shoulders of other's general abstractions: Fight software complexity with more software!
Workflow:
/.github/worflows/template.ymlFire unconditionally on any branch (Lines 1-6)
```yaml name: Initial commit workflow on: push: branches: - '**' ``` `name` is optional, it serves as a heading in the workflow execution. `on`- defines what should trigger this workflow. Since there is not an actual event for creating a new repo the hack is to use `'**'` this will make the trigger [fire on _any_ branch](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#patterns-to-match-file-paths). Obviously firing unconditionally on _anything_ may cause problems in the long run so this approach requires special measures, as you'll see laterAuthenticating
```yaml env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} ``` `env` is the section that defines environment variables. Setting `GITHUB_TOKEN` is the standard way of authenticating the GitHub CLI. GitHub actions support _secrets_ and GitHub automatically generate a token when starting a workflow, it's stored in the _secrets_ so this becomes that [standard way of using the automatic token authentications](https://docs.github.com/en/actions/security-guides/automatic-token-authentication)gitand theghCLI(Lines 8-9)Assigning permissions(Lines 11-15)
```yaml permissions: checks: write actions: write contents: write issues: write ```permissionsis used to change the permission if you need anything other than the default permissions. For this particular run I had to add explicit write permissions toactions(as you'll see later I'll disable this flow as soon as it has run once) andissuesso the workflow can create new issues and evencontentsto update the readme for the new repo coming off the template.Avoid multiple concurrent runs (Lines 17-19)
```yaml concurrency: group: "cpissues" cancel-in-progress: false ``` Due to the aggressive pattern `**` we could risk multiple jobs being triggered, dependent on enduser behaviour. But we can only allow one run at a time. and any new jobs can not be allowed to stop ongoing jobs. The default behaviour is to allow concurrency but it can easily be altered using the [concurrency features](https://docs.github.com/en/actions/using-jobs/using-concurrency).Define the image to use (Lines 21-24)
```yaml jobs: cpissues: name: "Initial commit only" runs-on: ubuntu-latest ``` `jobs` is where the action happens - the section defines all the jobs that the workflow knows of. Each job has it's on tag (you define the name yourself) in this case I only have one job and I came up with the clever name `cpissues`. `runs-on` specifies the runner and image that this workflow will run in. I only need to do small bits in the terminal, so I'm happy with the default _latest Ubuntu_ But even though `runs-on` is just a small clause it has a huge impact and by [defining your own runners and images](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idruns-on) you can open up a complete customized world to [_Platform Engineering_ and _Internal Development Platforms_](https://platformengineering.org/blog/what-is-platform-engineering) where you can configure the actions to support _anything as code_ - quite literally _"anything"!_ The finer details of the world that opens up underneath `runs-on` is beyond the scope of this tutorial. So go and explore yourself.Checkout the code (Lines 25-28)
```yaml if: ${{ contains(github.event.head_commit.message, 'Initial commit') }} steps: - name: "Checkout" uses: actions/checkout@v3 ``` `steps` defines the different steps executed in this workflow. Each step becomes an expandable/collapsable section in the job execution log you saw earlier, so defining smalle elegant steps is a means to keep a tidy and readable log of in your workflows. The rest of the this job happens in the context of these `steps` but before the workflow continues an `if` statement is used to fix the inconvenient fact that there is not a dedicated event that fires on repository creation only the greedy `**`which fires on any branch". The steps are simply ignored, unless the commit message of the commit spells exactly `'Initial commit'`. Because this is exactly the commit message the GitHub _always_ uses when it creates new repos off of templates. `uses` is referring to _actions._ Actions are called by their location on GitHub so consequently the GitHub Action `actions/checkout` resides on [https://github.com/actions/checkout](https://github.com/actions/checkout). And it uses the package release feature to offer a version `3`. The [@actions](https://github.com/actions) organization on GitHub is controlled and maintained by GitHub self. So you know, that if an action is located here it's considered GitHub standard. Almost every workflow will call this particular `actions/checkout` action as the first. It checks out the repo in the runner, and makes sure, that the rest of the job execution has access to the content of the repo.Cancel the workflow when it's done (Lines 30-38)
```yaml - name: "Raise semaphore (cancel workflow) or quit" run: | if $(gh workflow disable template.yml); then echo ::notice title=Workflow successfully disabled::This also works as a raised semaphore - parallel workflow runs will see it and cancel else echo "::warning title=Another job got here first::This room is crowded - I'm outta here!" gh api -X POST /repos/{owner}/{repo}/actions/runs/${{ github.run_id }}/cancel exit 1 fi ``` `run` is similar to `uses` but where `uses` calls a predefined action, `run` executes native shell commands. Starting the `run` clause with a pipe `|` followed by a new-line is a notation used to make the rest of the clause - a shell script. The first thing that happens is that the workflow `template.yml` is disabled. This is done simply by using the `gh` cli. The command will raise an error if it doesn't succeed - like if it's already disabled. The next line demonstrate how to [use simple `echo` statements to create messages in the workflow execution log](https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-debug-message) four levels are supported: `debug`, `notice`, `warning` and `error`. Perhaps you noticed the output from this earlier, when you browsed the content on the `Actions` panel.Copy issues over, using a
```yaml - name: "Copying issues over from template repo" run: | gh extension install thetechcollective/gh-cpissues gh cpissues thetechcollective/dx-intro --label template ``` Instead of having a lengthy script doing a lot of `gh`, `json`, `base64` and `jq` gymnastics to copy issues over - I put all of that nitty gritty stuff into a separate script and stored it as `gh` extension on [github.com/thetechcollective/gh-cpissues](https://github.com/thetechcollective/gh-cpissues). [`gh` extensions](https://docs.github.com/en/github-cli/github-cli/using-github-cli-extensions) works very much like regular `git` extension: If the script is named `gh-ghextension (Lines 40-43)Set the new README.md (Lines 45-43)
```yaml - name: "Set the README" run: | cp .github/template/README.md . rm -rf .github/template/ git config --global user.email "actions.bot@github.com" git config --global user.name "The GitHub Action Bot" git add -A git commit -m "Updated the README" git push origin ``` This bit uses `run` to replace the `README.md` in the root with the one that's designed, to be shown in the copy. This needs to be committed and pushed, and in order to do that we need to verify to git who we are by `name` and `email`. This creates a new commit - If you go and take a look, your repo already has two commits, but also notice that you only have one job execution - despite that the workflow subscribes to _any_ branch `'**'`: The disablement of the job worked as intended.