search

AmitKumarDas / metac

It is metacontroller and more
Apache License 2.0
57 stars 16 forks source link

Use semantic release in metac #64

Open grzesuav opened 5 years ago

grzesuav commented 5 years ago

To make automatic release notes and make easier to deploy new version, semantic-release could be used. https://github.com/semantic-release/semantic-release is a tool which :

It would need :

grzesuav commented 5 years ago

@AmitKumarDas I can try to add support (I have done it in my gitlab project ) probably shouldn't be hard for github.

AmitKumarDas commented 5 years ago

Sounds great @grzesuav

grzesuav commented 5 years ago

briefly looking on commit messages: 

8742540 [example] update steps to automate uninstall openebs (#62)
a238880 [example] update uninstall openebs example to latest metac (#61)
4ea7d56 [gctl] add UpdateDuringPendingDelete option (#60)
9af1eb8 [example] fix example on uninstall openebs (#59)
00b8375 [example] update rbac for uninstall-openebs example (#58)
a74a6fe [fix] name of finalizer assigned to watch resource (#57)
df70117 [example] update uninstall-openebs to latest metac (#56)
ee6f735 [config] fix loading of metac config file(s) (#55)
e6c273c [example] add updated metac to clean uninstall openebs (#54)
fd7e447 [example] add example to clean uninstall openebs (#53)
c624ab1 [readme] update readme with latest info about metac (#52)
3a0c517 [select] add advanced select rules to match specific resource(s) (#51)
59073f6 [config] add wait for all metac configs to load (#50)
ea19971 [example] improved README for set-status-on-cr example (#49)
cb54722 [example] update set-status-on-cr example with README.md & test.sh (#48)
ef8f6d5 [example] update set-status-on-cr to latest metac commit (#47)
0065a1c [fix] running metac from config (#46)
89bf818 [example] update set-status-on-cr to latest metac commit (#45)
d6ba051 [fix] config load by adding path along with file name (#44)
5c9a91b [example] update set-status-on-cr to latest metac commit (#43)
1c3c1a4 [local] fix config to avoid directories (#42)
eac4e63 [example] changes to set status on cr to point to latest metac (#41)
5df534c [local] add logs to local config based metac run (#40)
768f916 [example] set status on custom resource via inline hook (#39)
7c5d651 [start] move start code from main.go to start package (#38)
e24638c add generated client code to let metac be used as a lib (#37)
b625ffc [hook] add support to invoke inline functions as hooks (#36)
fb35f12 separate selector logic for watch & attachments of gctl (#35)
31c15db add skipReconcile to GenericController hook response (#34)
873ada4 [local] add option to run metac from local config (#33)
806e3a2 Adding a notice to the README about finalizers (#31)
d910577 adding a helmchart to deploy metac (#30)
6383e2c Make client-go throttling configurable. (#29)
41435e8 (tag: v0.1.0) [usecase, gctl]: showcase install/un-install of CRD (#28)

so the closes match is https://github.com/conventional-changelog/conventional-changelog/tree/master/packages/conventional-changelog-ember#ember-convention .

However I often find used Angular commit conventions (https://github.com/angular/angular.js/blob/master/DEVELOPERS.md#type) (at least up to types) which look likes :

chore(github): update Issue template with LTS info

I would propose the similar without the scope (int the ( ) brackets) :

chore: #555 - Update Issue template with LTS info

but it should be discussed and enforced in github checking for agreed commit messages format (like via https://github.com/probot/semantic-pull-requests or by some travis job check)

Commits up to now can be handled differently (like do not rewrite history, just map them somehow to major/minor/path/chore) and enforce format till this moment.

@AmitKumarDas any opinions on that ?

AmitKumarDas commented 5 years ago

chore: #555 - Update Issue template with LTS info

I guess above looks fine. I hope #555 is issue number. Issue may be optional perhaps.

Was #555 the PR number? If yes can it be at the last? I find most release use PR number at the last as a kind of suffix

AmitKumarDas commented 5 years ago

Can I set the old commits with GitHub labels like major, minor, chore etc. Will that do without getting too much into git stuff.

AmitKumarDas commented 5 years ago

@kmova can you please provide your thoughts as well.

grzesuav commented 5 years ago

Can I set the old commits with GitHub labels like major, minor, chore etc. Will that do without getting too much into git stuff.

No, I am afraid not. Semantic release relies only on git commit messages. Other option is to tag new release manually and enforce standards and introduce semantic release since this moment.

grzesuav commented 5 years ago

Moreover (https://github.com/semantic-release/semantic-release/blob/master/docs/support/FAQ.md#can-i-set-the-initial-release-version-of-my-package-to-001) they recommend to start from 1.0.0 so maybe it would be possible to manually tag a 1.0.0 with release notes etc and since then rely on commit messages ?

By the way @AmitKumarDas I have some questions :

  1. Which CI is used for doing build ? I do not mean test but actually to provide binary packages/docker images
  2. What are build artifacts ? Go binary/docker image ?
kmova commented 4 years ago

@grzesuav - thanks for bringing up these questions.

I agree we should go ahead and make a 1.0.0 release and start enforcing the commit message format.

Also, is there a way to enforce commit message format in github?

grzesuav commented 4 years ago

@kmova - yes, for format message it is actually an angular message format - https://github.com/angular/angular.js/blob/master/DEVELOPERS.md#-git-commit-guidelines . I would say that (scope) should be also optional - actually we only need type to make semantic-release work

for enforcing commit messages for example https://probot.github.io/apps/commitlint/

so, travis is also building go binaries and making and push image to docker registry ?

AmitKumarDas commented 4 years ago

@grzesuav Triggers have been set at quay to build docker images of metac whenever any PR is pushed against metac repo. These images are hosted at quay (similar to dockerhub).

One needs to use docker pull quay.io/amitkumardas/metac or quay.io/amitkumardas/metac as the image name. One can also use various tags present here

grzesuav commented 4 years ago

@AmitKumarDas ok, so on tag there will be tagged docker release made there ? Like quay.io/amitkumardas/metac:1.0.0 ?

AmitKumarDas commented 4 years ago

Yes. @grzesuav It's the same case when we cut a GitHub release on the repo. This release version is automatically reflected as a tag in the docker image.

grzesuav commented 4 years ago

hi @AmitKumarDas , maybe we can push this further ? We need to :

and then I can add semantic release (and automate relasign basing on it)

AmitKumarDas commented 4 years ago

Hi @grzesuav Do we need to first tag a 1.0.0 & then add the commit verification?

Do you have in mind any commit verificator that can be added to this repo right away? I can add the same or even accept any PR?

I shall do the manual release of 1.0.0 shortly. cc @luisdavim do you have any reservations on tagging.

grzesuav commented 4 years ago

Probably can add commit verification earlier, could try

luisdavim commented 4 years ago

We should also start publishing the helm chart to https://hub.helm.sh/ using the same version as the docker tag.

grzesuav commented 4 years ago

Initial config https://github.com/AmitKumarDas/metac/pull/104

grzesuav commented 4 years ago

@luisdavim yes, version should be sourced from git-tag. Release in semantic-release trigger new tag, which pushes docker image and helm chart release. Not sure @AmitKumarDas you are also publishing go package somewhere ?

grzesuav commented 4 years ago

after merging (or before) one of the maintainers should install https://github.com/apps/semantic-pull-requests - @AmitKumarDas can you do it ?

Configuration (types - like feat/fix/chore/etc) can be adjusted later (I will try to add semantic-release and config in one go)

grzesuav commented 4 years ago

For semantic release itself, it can be done via :

what is your preference ?

AmitKumarDas commented 4 years ago

For semantic release itself, it can be done via : github action (like https://github.com/marketplace/actions/action-for-semantic-release) travis ci what is your preference ?

Let us try with github actions. I have added NPM_TOKEN & GITHUB_TOKEN as personal access tokens. However, I am not sure of the level of access that needs to be granted.

AmitKumarDas commented 4 years ago

after merging (or before) one of the maintainers should install https://github.com/apps/semantic-pull-> requests - @AmitKumarDas can you do it ?

Done with merge as well as installing above app!

grzesuav commented 4 years ago

@AmitKumarDas https://github.com/AmitKumarDas/metac/pull/105

image

so seems working

grzesuav commented 4 years ago

I have played a bit, amending commit and editing PR title to semantic form makes it unblock the PR

grzesuav commented 4 years ago

https://github.com/AmitKumarDas/metac/pull/109

grzesuav commented 4 years ago

hi, example config there (and result of running on my fork). some remarks :

  1. every push to master will trigger a release
  2. currently tests are run on travis and independently github action triggers a release. Therefore master could fail and release could be trigered. Mitigation needed.
  3. release creates tag. Need to add triggering a docker build on tag. @AmitKumarDas how it is done currently ? I would imagine another github action can be added on tag to trigger releasing a docker image with correct version
  4. build artifacts - are they any ? Should we publish something when triggering a release, a binary or sth ? [I would imagine it would be useful]
  5. having changelog plugin would allow to have CHANGELOG.MD file (name could be customized) maintanable

any other suggestions ?

grzesuav commented 4 years ago

around 2. - possible solutions :

grzesuav commented 4 years ago

around 3.

AmitKumarDas commented 4 years ago

@grzesuav Any CI or github actions that makes the entire workflow smooth will be good. I would need help from your side to setup these.

Travis CI is used to check if integration test, unit test & docker image build is actually passing. Quay is setup from my own repository/login after integrating it with metac repo. Quay triggers build for every PR & merge to metac. Release is a manual tagging process that asks for a release version & upon setting triggers a build which in turn triggers Travis & Quay.

grzesuav commented 4 years ago

sure, I can do it. But wanted to confirm flow first :

  1. Run test via github actions on every push/PR- to check if it is possible
  2. When test pass on master - trigger a semantic-release, which creates also a tag
  3. When on tag - publish artifacts (if possible) and trigger docker push

@AmitKumarDas are you ok with that ?

grzesuav commented 4 years ago

I have found https://github.com/marketplace/actions/build-tag-publish-docker for building containers in github action

grzesuav commented 4 years ago

@AmitKumarDas I have integrated testing and releasing on single workflow : example : https://github.com/grzesuav/metac/actions/runs/58455222

tests and integrations-test are run in parallel, when both finish successfully, then (if on master branch) semantic-release will trigger a release

grzesuav commented 4 years ago

I believe now we can switch from quay-github integration and move build & push docker also to gitlab action.

@grzesuav Any CI or github actions that makes the entire workflow smooth will be good. I would need help from your side to setup these.

Travis CI is used to check if integration test, unit test & docker image build is actually passing. Quay is setup from my own repository/login after integrating it with metac repo. Quay triggers build for every PR & merge to metac. Release is a manual tagging process that asks for a release version & upon setting triggers a build which in turn triggers Travis & Quay.

So this will need to change as :

grzesuav commented 4 years ago

There is no manual action type on github, closest I could find would be repository dispatch event if I read this correctly : https://help.github.com/en/actions/reference/events-that-trigger-workflows .

More on that http://www.btellez.com/posts/triggering-github-actions-with-webhooks.html .

But actually I am not convinced that manual triggering gives us more benefits.

AmitKumarDas commented 4 years ago

sure, I can do it. But wanted to confirm flow first :

  1. Run test via github actions on every push/PR- to check if it is possible
  2. When test pass on master - trigger a semantic-release, which creates also a tag
  3. When on tag - publish artifacts (if possible) and trigger docker push

@AmitKumarDas are you ok with that ?

Yes

grzesuav commented 4 years ago

Found official https://github.com/marketplace/actions/build-and-push-docker-images#Example-usage docker plugin.

@AmitKumarDas could you encrypt in gitlab DOCKER_USERNAME and DOCKER_PASSWORD variables ?

Also, what is registry url for quay.io ?

grzesuav commented 4 years ago

Added changes to https://github.com/AmitKumarDas/metac/pull/109, however not sure how to test docker image push

AmitKumarDas commented 4 years ago

@grzesuav https://quay.io/repository/amitkumardas/metac

Should we push to some docker registry? We can create a new registry?

grzesuav commented 4 years ago

@grzesuav https://quay.io/repository/amitkumardas/metac

Should we push to some docker registry? Docker images must be in docker registry. I believe quay.io is docker registry

We can create a new registry?

It would be possible, although usually convention is to have repository/project so I think it could stay like it is.

As for me can try to use docker registry in form I configure it, and we will see how it works. I ca adjust it later with fix: commit which would just trigger minor release.

What do you think ?

AmitKumarDas commented 4 years ago

sounds great @grzesuav I am merging the PR.

AmitKumarDas commented 4 years ago

@grzesuav i am not sure of the credentials that need to be set against following ENVs

          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

I will look around to find the clues

AmitKumarDas commented 4 years ago

@grzesuav a new metac release got created. More additions are needed to following file: https://github.com/AmitKumarDas/metac/blob/master/.github/semantic.yml

For example more types needs to be added: https://github.com/angular/angular.js/blob/master/DEVELOPERS.md#type

grzesuav commented 4 years ago

@AmitKumarDas fix for docker release https://github.com/AmitKumarDas/metac/pull/116

seems separate workflow is needed to be triggered on tags

grzesuav commented 4 years ago

@grzesuav i am not sure of the credentials that need to be set against following ENVs

          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

I will look around to find the clues

GITHUB token is needed for semantic release to communicate with github API and :

NPM is not needed, it is for publishing for nmp repository, I will remove it in separate PR

grzesuav commented 4 years ago

@grzesuav a new metac release got created. More additions are needed to following file: https://github.com/AmitKumarDas/metac/blob/master/.github/semantic.yml

For example more types needs to be added: https://github.com/angular/angular.js/blob/master/DEVELOPERS.md#type

Yes, will look on that. Need to have same config for PR watcher and semantic release itself.

luisdavim commented 4 years ago

For tests I've started using these recently:

AmitKumarDas commented 4 years ago

For tests I've started using these recently:

https://github.com/bats-core/bats-core https://github.com/vincent-zurczak/detik https://github.com/jasonkarns/bats-mock

Hi @luisdavim above looks great. If we can integrate one of above with metac/examples & then run that on some K8s env (KIND or minikube) inside Travis or github actions then we achieve an automated e2e. Was that the intent of above post?

Meanwhile, I am working on a project around declarative e2e testing using metac. I shall talk about it once I am done.

luisdavim commented 4 years ago

I've been using k3d. What I have at the moment is very specific to the setup we use at my workplace, I'll see if I get some free time over the weekend and create something specific to metac.

grzesuav commented 4 years ago

hmm noting happened. I was testing it by making and pushing my tags, probably when tag is created on gitlab it happens without pushing ? Wil try to cover it today with different workflow enabler.

@luisdavim does bats is for tasting bash scripts itself ?