How to organize a CD process

[kirlat]

Our important goal is to automate a tedious process of releasing and deploying our builds. I think it makes sense to follow the best practices here. One important practice here is, I think, to embrace a Conventional Commits specification: https://www.conventionalcommits.org/en/v1.0.0/. It allows to formalize description of commits and automate tasks based on content of the commits.

Since Conventional Commits as an accepted standard, there are tools that can help with automation if commits are following that standard. One of those tools is semantic release. Semantic release

automates the whole package release workflow including: determining the next version number, generating the release notes and publishing the package.

Semantic release can be integrated with Travs CI pretty easily. The use of semantice release, Travis CI, and some other scripts allows to achieve a fully automated process such as:

• A commit is made with description following the conventional commits format.
• A PR for this commit is created on GitHub (manually).
• After PR is approved it is started to merge from a feature branch to a base branch. Base branches are: production, qa, master.
• Upon merge Travis CI starts the final testing round.
• If testing it successful, the following happens, depending into what branch the code is merged into:
  • If a base branch is master (i.e. this is a development build) version numbers in package.json, webextension's manifest and Safari config files are updated, and the build gets deployed to the development location (AWS S3 bucket). We can optionally tag it with a dev build tag that will specify a version number.
  • if a base branch is qa, the commit is tagged with a QA tag, version numbers in package.json, webextension's manifest and Safari config files are updated, a GitHub release is published with a QA code zip archive attached to it and, optionally, a code is deployed to a QA location
  • if a base branch is production, the commit is tagged with a production tag, version numbers in package.json, webextension's manifest and Safari config files are updated, a GitHub release is published with a production code zip archive attached to it and, optionally, a code is deployed to the production location

In order to make the process more in line with semver practices we can probably drop build numbers. If I understand correctly their only purpose is to distinguish different QA builds with the same version number. In order to fulfill this we can mark such build as prereleases: v1.3.3-qa.1, v1.3.3-qa.2, etc. This will follow semver guidelines and will be supported by all the tools listed above.

Please let me know what do you think about the approach described above. Do you see any problems with us implementing it? Can we make any improvements to it?

[kirlat]

We can also make Travis CI to check if the commit description follows the conventional commits guidelines by running a commit linter before any other tests are executed and stop the build if this check fails.

[irina060981]

A commit is made with description following the conventional commits format.

I belive it would need additional time. I create ussually much commits and they are not all very significant to use in ChangeLog.

We can also make Travis CI to check if the commit description follows the conventional commits guidelines by running a commit linter before any other tests are executed and stop the build if this check fails.

Do you meen that we would to to rewrite commits for build? I don't see direct linking here. We don't need to create commit for build. Do you suggest to to commit for every build?

Of course If it is a required part of development process, we should spend time for this. And for creating tests, and for creating documentations too.

I believe that it is of course useful but would make development process a little more "bureaucratized".

[kirlat]

A commit is made with description following the conventional commits format.

I belive it would need additional time. I create ussually much commits and they are not all very significant to use in ChangeLog.

This is a good point. If we're using a lot of "intermediary" commits that do not introduce new finalized functional code changes but are rather a work in progress then, I think, a requirement to make every commit match a conventional commits structure will not be feasible.

[kirlat]

We can also make Travis CI to check if the commit description follows the conventional commits guidelines by running a commit linter before any other tests are executed and stop the build if this check fails.

Do you meen that we would to to rewrite commits for build? I don't see direct linking here.

No, my idea was that it will check if the commit format matches the standard and if not it will fail a Travis build. No rewrites (as ESLint does now), just validation. I don't think it's possible to make meaningful commit rewrites because if some information is missing from the commit there is no place where a linter would gather it to fix the issue.

But if we decide not to follow conventional commits then we will not implement this.

[kirlat]

We don't need to create commit for build. Do you suggest to to commit for every build?

We can go different ways about that and I'm not sure which one is best for us. But I was suggesting not to create a specific commit for each build (we can do that too if we would like to) but to use the one that GitHub creates automatically for each merge: Commit F is created by GitHub automatically even if E and F are the same (please correct me if I'm wrong here). So F is in fact the build (or the release) commit. I was thinking about tagging it automatically and deploying code from it. Or, if working with F prove to be hard, we can employ E, with the condition that E and F are the same (i.e. if changes from master are fully merged into a feature branch before the merge of the PR).

There are several ways to implement all this successfully. I'm not sure which one would be the best for us. That's I think we should start with the changes only after a thorough discussion.

[kirlat]

I believe that it is of course useful but would make development process a little more "bureaucratized".

I'm not a fan of bureaucratization too. So I think we should find the simplest, minimal approach that will solve our current inconveniences. To me areas that creating most troubles are:

• A need to manually create "distribution" archives, such as archives with the code we give to QA or upload to webestensions' stores.
• A need to manually upload code to remote servers for testing and production
• A need to synchronize different releases. For example, if we add some new functionality to components, we need to rebuild components, upload it to npm, rebuild webextension, submit it to the store, rebuild embed-lib, updating a link to the new npm release in the process, submit it to CDN, rebuild Nemo UI and upload it to remote. And not to forget to update versions appropriately.
• It's hard to increase versions consistently, because version increases can be made in a feature branch before it is merged to master or in the master after the merge. There can be two or more feature branches existing at the same time each having its own version increased in some way and how would we come up with the correct final version after these two are merged at almost the same time?

So I think the following points are important in solving this:

• When we want to release (by release a mean a complete build with some finalized piece of functionality that is merged into either master, qa, or production and should be deployed to the development, QA, or production environment correspondingly) we should somehow indicate that, either with the conventional commit message, or by simply adding a [release] text or anything like that to the commit message, or running some script locally.
• When the release system (by that I mean a combination of GitHub, Travis, and whatever other tools we will use) will detect a release and a PR is merged it will automatically do all described above, depending on into what base branch ( master, qa, or production) it is merged into.

That, on my opinion, would be ideal.

[kirlat]

One of the simplest way to implement this would be, I think, to have a local script that will take care of a version increase and merge a base branch into a feature branch before merging a PR.

With this, the merging sequence would be:

1. Wait for PR to be approved
2. Fetch the latest version of a base branch (a branch to which the changes are merged)
3. Merge a base branch into a current feature branch.
4. Run a local release script. It could be a command like release patch, release minor, etc. where the second word indicates what type of changes were made and what part of a version number shall be increased. Based on that, a script will increase a version number in package.json, and, for webextension, in manifest and Safari plists.
5. Commit the changes related to the version update.
6. Merge a PR. Upon merge, depending on the base branch, Travis will create a release (if necessary) and will deploy files to the corresponding location (if necessary).

[kirlat]

On my opinion, we shall probably increase versions for "development releases" too (i.e. when we merge changes from a feature branch to master), because otherwise it will be impossible to keep track of all the features that were added before a build goes to QA. As a result of this, it will be impossible to set the version number that will reflect the total amount of changes introduced.

I also think it might be important to have an actual version number during dev testing to understand what version (and thus what feature set) we're working with.

Let's say we have a package with version 1.0.7.

• We added a fix and merged it to master. The version became 1.0.8-dev. I think it is good to have a dev postifix to distinguish a development version from a production one.
• We added a minor feature and merged it to master. The version changed to 1.1.0-dev.
• After another fix the version became 1.1.1-dev.
• We decided to release the code in its current state and put it to QA. The version became 1.1.1-qa.0. The version number is not increased as no new features were added.
• QA found bugs and we issued a fix. The version became 1.1.1-qa.1.
• QA approved and we merged into the production branch. The version became 1.1.1. At that moment of time, as the new features were added to the dev branch (i.e. master) as the QA was testing the version at master might be 1.2.3-dev.

@balmas, @irina060981: would that make sense to you?

[balmas]

QA approved and we merged into the production branch. The version became 1.1.1. At that moment of time, as the new features were added to the dev branch (i.e. master) as the QA was testing the version at master might be 1.2.3-dev.

I'm not sure I understand what you are suggesting with this point about 1.1.1 vs 1.2.3-dev. I think you are saying that at the point a qa branch is merged into master, master might already be well beyond the version of the qa branch. Is that correct? If so, how would that be handled under this proposal?

[balmas]

On my opinion, we shall probably increase versions for "development releases" too (i.e. when we merge changes from a feature branch to master), because otherwise it will be impossible to keep track of all the features that were added before a build goes to QA.

I do agree with this.

[balmas]

One of the simplest way to implement this would be, I think, to have a local script that will take care of a version increase and merge a base branch into a feature branch before merging a PR.

With this, the merging sequence would be:

1. Wait for PR to be approved
2. Fetch the latest version of a base branch (a branch to which the changes are merged)
3. Merge a base branch into a current feature branch.
4. Run a local release script. It could be a command like release patch, release minor, etc. where the second word indicates what type of changes were made and what part of a version number shall be increased. Based on that, a script will increase a version number in package.json, and, for webextension, in manifest and Safari plists.
5. Commit the changes related to the version update.
6. Merge a PR. Upon merge, depending on the base branch, Travis will create a release (if necessary) and will deploy files to the corresponding location (if necessary).

On the whole, I agree with this. It is essentially what I try to do now manually, but am not always consistent about. So having some formalization of the procedure would be good. Ultimately I think we could automate it but agreeing upon a procedure that is scripted and manually executed would be a good start.

[balmas]

I do think trying to follow conventional commits would be more than I want to undertake just yet.

[balmas]

I think whatever we come up with, the following requirements apply:

1) we need to be able to identify exactly which version of any dependency is included in any development, qa, or production build. Identifying the alpheios-components version is not sufficient, because it has many dependencies. 2) we need a reliable way to freeze the version of a dependency, both for alpheios-produced packages and 3rd party packages 3) we need a clear process for development that tells us when to increase the version of a package and how

[kirlat]

QA approved and we merged into the production branch. The version became 1.1.1. At that moment of time, as the new features were added to the dev branch (i.e. master) as the QA was testing the version at master might be 1.2.3-dev.

I'm not sure I understand what you are suggesting with this point about 1.1.1 vs 1.2.3-dev. I think you are saying that at the point a qa branch is merged into master, master might already be well beyond the version of the qa branch. Is that correct? If so, how would that be handled under this proposal?

Yes, that's exactly that. Master (i.e. a dev branch) will always be ahead of production in features, and correspondingly, in a version number.

Let's continue a timeline of the scenario referred above:

• Dev version is 1.2.3-dev and prod version is 1.1.1.
• We added some new features to dev and it became 1.3.1-dev. The prod stays at 1.1.1 because we have not merged anything into it yet.
• We decided to release the current state of dev. Dev code goes to QA and becomes 1.3.1-qa.1. Prod is 1.1.1 still.
• QA approved the changes and we merged them to prod. Then production will be at 1.3.1 version now while master will probably be ahead of it.

Would that work for us?

[kirlat]

2\. we need a reliable way to freeze the version of a dependency, both for alpheios-produced packages and 3rd party packages

Would that be useful if we automatically freeze all dependencies by updating package.json once the build goes to QA and then to production?

[balmas]

• Dev version is 1.2.3-dev and prod version is 1.1.1.
• We added some new features to dev and it became 1.3.1-dev. The prod stays at 1.1.1 because we have not merged anything into it yet.
• We decided to release the current state of dev. Dev code goes to QA and becomes 1.3.1-qa.1. Prod is 1.1.1 still.
• QA approved the changes and we merged them to prod. Then production will be at 1.3.1 version now while master will probably be ahead of it.

Would that work for us?

I think so.

[balmas]

2\. we need a reliable way to freeze the version of a dependency, both for alpheios-produced packages and 3rd party packages

Would that be useful if we automatically freeze all dependencies by updating package.json once the build goes to QA and then to production?

Yes!

[irina060981]

Let's continue a timeline of the scenario referred above: Dev version is 1.2.3-dev and prod version is 1.1.1. We added some new features to dev and it became 1.3.1-dev. The prod stays at 1.1.1 because we have not merged anything into it yet. We decided to release the current state of dev. Dev code goes to QA and becomes 1.3.1-qa.1. Prod is 1.1.1 still. QA approved the changes and we merged them to prod. Then production will be at 1.3.1 version now while master will probably be ahead of it. Would that work for us?

I have a question, With this workflow how would we define - what production branches === development branches.

For example, we lost some feature (similiar to not long ago situation), we could see that we have in production the following production branches. How could we found - what development branches were before? Some of thems could be not merged if qa was not successfull for example, I believe/

[kirlat]

I have a question, With this workflow how would we define - what production branches === development branches.

For example, we lost some feature (similiar to not long ago situation), we could see that we have in production the following production branches. How could we found - what development branches were before? Some of thems could be not merged if qa was not successfull for example, I believe/

I think we shall tag all the merges and we can use tags to see what went from where. Let's say we started development of a new feature. We created a feature branch where will be all our development commits for that feature. Once we've done with development, we merge the feature branch to master and mark the merge commit with a v1.3.2-dev tag (let's say 1.3.2 is the current version).

Let's say there is another feature developed at approximately the same time and it is merged to master a bit later. The merge commit will be tagged as v1.4.0-dev.

When it goes to QA, we take everything what's in the master up to date (1.4.0) and merge that to a qa branch. The merge commit will be tagged as v1.4.0-qa.0. If there are any fixes required, we'll merge them into the qa branch and tag as v1.4.0-qa.1, v1.4.0-qa.2, etc. Once QA accepted the changes, we do a merge from qa to production and tag a merge commit as v1.4.0.

After that we merge production to master to add all the fixes made during QA. I'm just not sure how to tag that merge.

If a previous merge to production was made at 1.3.0, we will know that all the versions between 1.3.0 and 1.4.0 compose the changes that were introduced into production with this commit. So if something is broken, we can go to master and examine all the tags between 1.3.0 and 1.4.0. Those tags well be end points of all feature branches that lead to individual development commits. From there we will be able to go into individual commits that comprise the feature development chronology and examine them in details.

Would that be enough for us?

[balmas]

If a previous merge to production was made at 1.3.0, we will know that all the versions between 1.3.0 and 1.4.0 compose the changes that were introduced into production with this commit. So if something is broken, we can go to master and examine all the tags between 1.3.0 and 1.4.0. Those tags well be end points of all feature branches that lead to individual development commits. From there we will be able to go into individual commits that comprise the feature development chronology and examine them in details. Would that be enough for us?

Seems so. I think the problem we had with lost commits before was probably due to my committing directly to master, rather than using a PR. I think having a defined process that we all (including me!) follow will help prevent that.

[balmas]

this practice is implemented and documented in the README.