RFC: Continuous Delivery

giorgiosironi commented 6 years ago

Problem

Libero needs continuous feedback on pull requests and release candidates, in terms of:

Style checks, static analysis and similar code-oriented tools being run
Containers and other artifacts being built successfully
Project, end-to-end (for components declared compatible by semantic-versioning), performance, etc. tests run successfully
Integration with multiple components versions (e.g. Python, PHP, Symfony, …)
Deployment to key target platforms being successful (e.g. container-based or not; GKE, EKS, AKS, on-premise)

The maintenance requirements are:

Being able to modify a build definition from source control (some Git repository) to foster contributions and reviews
Due to the microservice orientation, being able to extract tools and build steps from different projects to reduce duplication
If not necessary to run custom servers, avoiding operations work by outsourcing management of the infrastructure

Suggestion

Provide Travis CI builds for all projects, including both code and infrastructure needed to test it.

Concerns

Limited capability to customize the environment in which builds run, requiring tools to be installed on the fly on every build that needs them.
Duplication of build patterns and steps across different repositories is difficult to remove. Build files generation may be able to mitigate this.
The level of performance provided is not under our control, both in terms of underlying resources of a single build, reuse of resources created by previous builds, and number of concurrent builds that can be run.

Other evaluated solutions (comparison table):

Circle CI is less popular than Travis CI but has a promising model for customization of build environments (container-based). It has however a setup using remote Docker that makes it difficult to build with docker-compose e.g. mounting local volumes.
Jenkins and TeamCity are at their core unmanaged solutions oriented to private projects, which create the need for maintenance, opening and securing of a new build platform.
GoCD is subpar in the Pipeline as code area, requiring manual UI configuration or non-standard solutions to manage build definitions through source control.
Concourse is a niche product, still immature and tied to the Pivotal ecosystem.

stephenwf commented 6 years ago

I've got 2 main questions about the CD.

1) For the end-to-end testing, do you envision this being something declared completely in the source control of the micro-service? Like for example, bootstrapping services X, Y and Z with fixtures A, B, C. Also will this put micro-services in a position where they may be tightly coupled to other versions of components, or would there be a fixture matrix of compatible versions?

2) When artefacts are built, is the assumption that they will be delivered to GitHub or a package manager or somewhere else? (I know the exact destination may not be in the scope of this)

giorgiosironi commented 6 years ago

For the end-to-end testing, do you envision this being something declared completely in the source control of the micro-service?

See next answer for how to declare this, but in general in end-to-end scenarios a target platform with some defined infrastructure needs to be defined first (e.g. deploy on GKE or deploy on AWS). So there may be some out of band information in that environment that is not completely under the control of the microservice. Incidentally, this puts the result of a end-to-end test not purely in the hands of the microservice being tested but also in the state of the outside world.

Also will this put micro-services in a position where they may be tightly coupled to other versions of components, or would there be a fixture matrix of compatible versions?

It will depend a lot on downstream usage, but the bare minimum is testing the current master branches work together. In addition to that and with compatibility and decoupling in mind, we could also test version 1.2.3 of service X works with 1.0.0 of Y and 1.0.0 of Z. This is inspired by our lowest library testing mode but it's a different context, relying on the semantic versioning convention. Needs to trialed to see if it works in practice, and considering we will be on 0.x versioning for a while.

giorgiosironi commented 6 years ago

When artefacts are built, is the assumption that they will be delivered to GitHub or a package manager or somewhere else? (I know the exact destination may not be in the scope of this)

So far it looks like we support at least container images to deliver a service wherever it is being deployed. These would be pushed to Docker hub for simplicity. If we support other kinds of deployments other artifacts would have to be produced and put somewhere public as well.

smeall-zz commented 6 years ago

No objections from us to adopting Travis CI.

BlueReZZ commented 6 years ago

Travis CI is now in use and working well. We've recorded the decision here as an ADR in https://github.com/libero/community/pull/16 and https://github.com/libero/community/blob/develop/doc/adr/0002-continuously-delivery-travis-ci.md

libero / community