humanmade / altis-dev-tools

Dev Tools module for Altis
https://www.altis-dxp.com/resources/docs/dev-tools/
7 stars 2 forks source link

Provide GitHub Actions sample config within docs #177

Open shadyvb opened 3 years ago

shadyvb commented 3 years ago

Related to https://docs.altis-dxp.com/nightly/dev-tools/continuous-integration/

We recommend, and generate config for, Travis as the recommended CI env, is there something that prevents us from suggesting GitHub Actions as an alternative CI env, with basic docs and examples ?

roborourke commented 3 years ago

Only thing preventing us is time and capacity, at the time GH actions weren't really suitable but I believe @joehoyle got it working and @kadamwhite too.

It's partially related to https://github.com/humanmade/altis-dev-tools/issues/102 and also the RFC for having a single dev dependency module for the local environments here https://github.com/humanmade/product-dev/pull/561

Ideally I'd like for devs to be able to choose their preferred CI environment by either using documented config or a scaffolding command like composer dev-tools scaffold travis or composer dev-tools scaffold github. I thought I'd captured that in an issue somewhere already but I guess not!

kadamwhite commented 3 years ago

I posted about this on our internal dev list, so cross-posting the relevant content here.

The goal is to adapt Altis' composer dev-tools phpunit command to GitHub Actions. I turned first to John's testing workflow for the Authorship plugin, which provides a great baseline. However, the composer install step in that action didn't work for the Altis project I was working on because the project pulls in multiple private GitHub repositories via composer. Most of the complexity in the post below is due to private Composer VCS dependencies, and to a TTY mode issue in Local Server.

Post about how I set up GH Actions on Altis ### Install private composer dependencies The [`php-actions/composer` action](https://github.com/php-actions/composer) supports private repositories by letting you [inject an SSH key using GitHub Secrets](https://github.com/php-actions/composer#installing-private-repositories). We followed this process: - Create a new SI user GitHub account within the team's organization. - Give that account read-only access to all the relevant dependency repositories. - Generate a _passwordless_ SSH key and [add it to the user](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account). (I tried using a key with a passphrase and spent a whole day wondering why it wouldn't work; d'oh) - [Add that SSH key as a GH secret](https://docs.github.com/en/actions/reference/encrypted-secrets) in the settings for the main project repository - Update the GitHub Action workflow file to look like this: ``` name: Unit Tests on: push: branches: - 'staging' - 'main' pull_request: branches: - '**' jobs: phpunit: strategy: matrix: php: ['7.4'] name: PHPUnit on PHP ${{ matrix.php }} runs-on: ubuntu-18.04 steps: - name: Checkout repository uses: actions/checkout@v2 - name: Cache Composer dependencies uses: actions/cache@v2 with: path: /tmp/composer-cache key: ${{ runner.os }}-${{ hashFiles('**/composer.lock') }} - name: Install PHP dependencies uses: php-actions/composer@v6 with: version: 1 # Composer 1 php_version: ${{ matrix.php }} ssh_key: ${{ secrets.ssh_key }} ssh_key_pub: ${{ secrets.ssh_key_pub }} ``` We'll be switching over to Composer 2 soon, but for now this is the most straightforward way I could find to install Composer packages in GH Actions. The `php-actions/composer` action will ensure PHP is set up, so we don't need to manually install that as John did in Authorship. I've also copied the caching example from the composer action's documentation (although I haven't seen it actually speed things up at all). ### Running PHPUnit in Docker `composer dev-tools phpunit` executes the unit tests inside the running container. So, let's run Docker! ``` - name: Start the containers run: | composer server start ``` Whoops. Job failed: `vendor/docker-compose.yml` is not writable, so the generation of the configuration file fails. OK, the Composer action must install as root. Composer doesn't like being run with `sudo`, so let's see if that action has an option to set the user... nope, doesn't seem like it. We'll just manually make the file writable instead: ``` - name: Start the containers # Make the docker-compose file writeable, then start the server. run: | sudo touch vendor/docker-compose.yml sudo chown $(whoami) vendor/docker-compose.yml composer server start ``` Commit, push, and... heck. ``` TTY mode requires /dev/tty to be read/writable. ``` 😬 I have no idea what that means. I can search the web, though! Hmm, alright, every article seems to want me to run `set(‘git_tty’, false);`. But, I don't want to have to modify Local Server. Let's do some research. _A ha!_ There's [a post about this specific TTY issue in the context of GitHub Actions](https://bastide.org/2021/01/04/docker-compose-and-not-able-to-set-a-tty-in-a-git-hub-actions/). It's a known issue, and there's a standard workaround! Let's use `script` to run the `composer server` command in a "typescript" of a terminal session. That doesn't have anything to do with TypeScript: it means that only a log of the session gets passed back from the command, so we sidestep the lack of TTY. ``` - name: Start the containers shell: 'script -q -e -c "bash {0}"' # Make the docker-compose file writeable, then start the server. run: | sudo touch vendor/docker-compose.yml sudo chown $(whoami) vendor/docker-compose.yml composer server start ``` Awesome, now the containers are pulling! And... the action still fails. The containers start, but it crashes out afterwards. The install was succeeding, but the error was instead triggered while trying to set up ElasticPress. We don't need ES to run our tests, so 🤫 let's just add `continue-on-error: true` to our command, and move on. _n.b._ Rob has pointed out in a Slack thread that we could have solved this by setting the environment variable `ES_MEM_LIMIT=2g`. There are [Altis docs about that issue](https://docs.altis-dxp.com/local-server/troubleshooting/#elasticsearch-service-fails-to-start). Anyway, now the containers are starting, and we can finally run our `dev-tools` command! We have the same file permissions issue when running PHPUnit, because the `dev-tools` command creates `vendor/phpunit.xml`. I used the same approach as above to make that file individually writable, but in retrospect I could also have made the CI `runner` user the owner of the `vendor` directory itself. ### Putting it all together This is our complete PHPUnit action: ``` name: Unit Tests on: push: branches: - 'staging' - 'main' pull_request: branches: - '**' jobs: phpunit: strategy: matrix: php: ['7.4'] name: PHPUnit on PHP ${{ matrix.php }} runs-on: ubuntu-18.04 steps: - name: Checkout repository uses: actions/checkout@v2 - name: Cache Composer dependencies uses: actions/cache@v2 with: path: /tmp/composer-cache key: ${{ runner.os }}-${{ hashFiles('**/composer.lock') }} - name: Install PHP dependencies uses: php-actions/composer@v6 with: version: 1 # Composer 1 php_version: ${{ matrix.php }} ssh_key: ${{ secrets.ssh_key }} ssh_key_pub: ${{ secrets.ssh_key_pub }} - name: Start the containers shell: 'script -q -e -c "bash {0}"' # Make the docker-compose file writeable, then start the server. run: | sudo touch vendor/docker-compose.yml sudo chown $(whoami) vendor/docker-compose.yml composer server start continue-on-error: true # ElasticPress may report failure - name: Run PHPUnit run: | sudo touch vendor/phpunit.xml sudo chown $(whoami) vendor/phpunit.xml composer dev-tools phpunit ``` Once I got all of this working, Rob also clued me in to the fact that Joe had gone through a similar process on another codebase. Instead of using the `script` trick, Joe's action uses `sed` to live-patch Local Server within the CI environment. Fixing things more permanently in Local Server would probably be a prerequisite to solving this ticket generally, because the more we can remove the confusing bits, the easier a scaffold command will be.