Increase automated test coverage for GitLab and Pipelines

[ataylorme]

The tests for BitBucket and GitLab only run build:project:create.

The GitHub tests run more robust coverage, testing creating pull requests, watching for the multidev, merging the PR, etc.

The main difference is the lack of a script to watch the results of jobs on GitLab or Pipelines. The CircleCI CLI provides watch functionality out of the box.

We should have more robust tests for GitLab and BitBucket Pipelines with parity for coverage provided by the CircleCI coverage.

[ataylorme]

We are using the CircleCI CLI and it's watch command to determine if CircleCI jobs are done, pass, fail, etc.

@danielbachhuber is going to do some research to determine if there is a similar command-line tool for BitBucket and/or GitLab and, if not, what work would be needed to get similar functionality from querying their APIs directly.

[philltran]

Looks like bitbucket pipelines has an after-script that can run a script after build completed, fails etc. https://bitbucket.org/blog/after-scripts-now-available-for-bitbucket-pipelines

[ataylorme]

Thanks for the link @philltran. Thinking "out loud" here, we could possibly:

• Have a job in CircleCI that starts a test job in BitBucket Pipeline
• Hold the CircleCI workflow for manual approval
• Once the BitBucket Pipelines job passes, call the CircleCI API to clear the manual approval
  • Or, if the BitBucket Pipelines job fail, mark the CircleCI job failed.

[philltran]

@ataylorme is the goal to still have all the CI handled and tests run on BItbucket PIpelines? Or will CircleCI still control the flow and just tests are run on BitBucket Pipelines?

[ataylorme]

@philltran the goal is to have parity with test coverage for what is going on with the GitHub + CircleCI tests for BitBucket and GitLab as well.

Right now, the tests for the Build Tools project itself are all run in CircleCI and I expect to keep the test pipeline for Build Tools source code in CircleCI.

That means expanding the test coverage for BitBucket and GitLab within our current CircleCI testing workflow.

[danielbachhuber]

@ataylorme Both Bitbucket and GitLab have API endpoints for pipelines:

• https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/pipelines/%7Bpipeline_uuid%7D
• https://docs.gitlab.com/ee/api/pipelines.html#get-a-single-pipeline

In both cases, I expect we could do something similar to watch-circle.sh: loop API requests until we see whether the pipeline passed or failed.

[ataylorme]

In both cases, I expect we could do something similar to watch-circle.sh: loop API requests until we see whether the pipeline passed or failed.

Yes, that is my expectation. After chatting with @greg-1-anderson, calling the CircleCI API from BitBucket or GitLab in post-job hook would require sharing a CircleCI token with the test projects on those services, which we want to avoid.

[danielbachhuber]

@ataylorme Ok. Do you want me to move forward with implementation over the next week, or should I wait?

[ataylorme]

@danielbachhuber please proceed with implementation. Let me know if you have further questions or run into blockers.

[ataylorme]

@danielbachhuber just noting from our call that the tests should make use of terminus project:pull-request:create from the Terminus CLU plugin, which is already installed on the Dockerfile that CircleCI runs, instead of making direct API calls or using hub.

[danielbachhuber]

@sugaroverflow I think this one is a little bit more important to include in 2.0 but not critical. From what I recall, it's pretty much done and just needed a final review https://github.com/pantheon-systems/terminus-build-tools-plugin/pull/305#issuecomment-547071216

I'm not sure why the last two merge commits on that PR are failing though. This might be something we need to work through together.