shipwright-io / build

Shipwright - a framework for building container images on Kubernetes
https://shipwright.io
Apache License 2.0
667 stars 113 forks source link

As developer, I 'd to kick off build to generate a image with new tag, so that to do image version management #150

Open ZhuangYuZY opened 4 years ago

ZhuangYuZY commented 4 years ago

The basic build flow from developer I think:

  1. define build (build cr), developer provides git source, build strategy, output image registry url.
  2. run continue build (buildrun cr) to generate images, with specific tag, 0.1.0, 0.2.0 ...
  3. run deploy pipeline to deploy application with the full image url(+tag), which could be get from cr status.

To implement above scenarios, what I understand there is some gap here. for 2, we have to update build cr spec output with new tag each time, then create a buildrun cr to kick off a build. One build needs to touch 2 cr which is not good from end user perspective. for 3, we have to get image+tag from build cr spec, which is not good, as spec is not design as result, status is for result.

Proposal: Add tag in buildrun CRD spec, end user to specific image tag when create buildrun. Add image in buildrun CRD status, where end user can get a full url of image when build completed successfully.

@sbose78 @zhangtbj your thought ?

SaschaSchwarze0 commented 4 years ago

I generally like this proposal. But does it then mean that a tag MUST NOT be included in the output image in the build cr ? And we therefore should therefore "fail" a build cr which tries to specify it ?

zhangtbj commented 4 years ago

In our PoC, our design like this:

We just set the image url in the spec.output.image, like docker.io/myorg/myimage

And we provide a TagPolicy like Latest or ByCount,

Then, when we generate the taskrun and the output image for the taskrun, we will add the related tag in the output of taskrun, like this: docker.io/myorg/myimage:v1 if end user choose ByCount policy.

If end user chooses Latest policy, we keep using latest for the tag.

And then, we show the image status in the buildRun or image CR status, to show its tag. So it looks like:

...
spec:
  tagPolicy: "ByCount"
  output:
    image: us.icr.io/knative_jordan/kaniko-taxi

...

...
status
  image: us.icr.io/knative_jordan/kaniko-taxi:v1

Or similar style or configuration.

BTW, we also have a similar policy for auto-trigger. Just FYI, hope it is helpful :)

ZhuangYuZY commented 4 years ago

@zhangtbj I know before we working build-v2, we have POC for auto generate tag. But may not for first phase.

For first phase, I suggest to just have capability for end user easy put the special tag string, like 0.2.0 in buildrun, then tag the image. Then we can iterate to have auto generate tag in future.

Thank you.

sbose78 commented 4 years ago

Thinking out a loud: Maybe we should add a configuration to auto generate unique tags?

sbose78 commented 4 years ago

Yes, similar to the tagPolicy approach.

ZhuangYuZY commented 4 years ago

Yes, auto generate unique tag is good feature, we can do semver version policy I think. And please also consider how end user could bump a version of image tag, like for 1.0 release, image version could be 1.0.x, bump to 1.1 release, then version could be 1.1.x, future may be 2.0.x, 3.1.x. And another case, end user may have own tag policy, we may not support all policies. We may also need to add tag in buildrun CRD spec.

otaviof commented 4 years ago

On OpenShift is common to use the image SHA signature as tag, by the way.

otaviof commented 4 years ago

Thinking out loud, we could use the Git project itself to define the version from different places:

otaviof commented 4 years ago

Community notes:

zhangtbj commented 4 years ago

Hi all,

I am not sure if we can do this feature step by step.

And first of all, I think we can add output.image property in BuildRun, so that the end-user can overwrite that config to the output.image in build CR, then build and push a new image or new tag of the image by using the buildRun and without modifying/editing the parent Build definition.

Do you agree with that?

zhangtbj commented 4 years ago

Hi all,

If no other comment, I will start the phase 1 of this feature above (add output.image property in BuildRun) ^^ tomorrow, so that we complement our capability first.

Thanks!

sbose78 commented 4 years ago

Created this to be track the tag policies we are interested in https://github.com/redhat-developer/build/issues/171

sbose78 commented 4 years ago

Hi Jordan, are you suggesting that the user might specify one image in Build and another in BuildRun?

zhangtbj commented 4 years ago

Hi Shoubhik,

Yes.

For different buildRun or rerun, we don't have the capability to build the different tag of image somewhere. Now what we have to do is:

It is still complex and not convenient for end user.

And for the tag policies (#171), I think we still need more discussion and maybe one or two month to confirm and implement that.

So I think we can use output.image in buildRun to overwrite the build setting first.

Or do you other suggestion or concern? :)

I will stop here until ALL of us agree it is good for us.

sbose78 commented 4 years ago

think we still need more discussion and maybe one or two month to confirm and implement that.

Yes! No hurry, let's brainstorm and design it well.

When we do have tag policies, would the BuildRun output.image tag still be relevant?

zhangtbj commented 4 years ago

I think it may not relate to the tag policy, or buildRun configuration will overwrite the build policy?

But I think we should have this requirement that an org of users define a build as team work. For each buildrun, I use my serviceaccount(my docker account) maybe to build and push to the test image or my registry.

If we define an output.image url in build. like docker.io/production/application, I think maybe not all users have the secret to push the image to the production registry.

otaviof commented 4 years ago

Community notes:

otaviof commented 4 years ago

As a future tag policy, I also would like to be able to define the image tag based in the project being build. Like re-use Git tag as container tag, and the same for commit-id/branch, similarly than quay.io does.

zhangtbj commented 4 years ago

Yep, I think we can provide more image tag policies for us:

At least, I think we should allow end-user to manually change to special image tag (The By Manual way)

And I think what we can do for By Manual way now, is we can have the output.image setting in buildrun, so that the end-user can manual set the image tag in output.image of buildrun and build a new image tag.

If end user chooses By auto-generation version or By Commit id image tag policy in future. The output.image setting in buildRun will be ignored.

Like the current ServiceAccount setting in buildRun:

I am not sure if it is ok for all of us? Or any other suggestion?

ZhuangYuZY commented 4 years ago

Yes, I agree with above.

For me, build like build-template which could shared in a team and buildrun could overwrite some property in build template when kick off a build, like branch and image tag.

Consider from end user experience, a development team define a build to build an image officially, like build from master branch and with auto tag policy. It works fine normally. But sometimes, one developer wants to do a special build to test some issue locally, like build from 'bug-fix' branch and with tag of bug-number.

To do this, if buildrun could accept parameters to overwrite build, that will be easy to address above case. If we ask to change build to build from 'bug-fix' branch and with tag of bug-number temporally and create buildrun. It will break the sharing of build, because at same time, other can also create buildrun, but result is not expected.

And pattern of Template and TemplateRun allow overwrite also using by Tekton Task and TaskRun.

So suggest to allow overwrite in 'buildrun' API spec. Thank you.

@sbose78 @otaviof your thinking ?

otaviof commented 4 years ago

Yep, I think we can provide more image tag policies for us:

* By auto-generation version

* By Commit id

* By Manual

Good, agreed! And to add here, git tag should also be available, by the way.

Thinking out loud, maybe we should also allow the user to inform a command which will return the version of the project, in other words, which tag the image should have, in the project folder itself. Similarly to informing the location of the Dockerfile, but this time with the notion of which command should it run in order to obtain the version current.

At least, I think we should allow end-user to manually change to special image tag (The By Manual way)

And I think what we can do for By Manual way now, is we can have the output.image setting in buildrun, so that the end-user can manual set the image tag in output.image of buildrun and build a new image tag.

Definitely, well said.

If end user chooses By auto-generation version or By Commit id image tag policy in future. The output.image setting in buildRun will be ignored.

Okay, fair enough. We should also inform about this circumstance in the status.

zhangtbj commented 4 years ago

Yes, agree. We should suppport both UI and cli.

And we should design the tag policy logic first then prioritize the policies and do it one by one. Then also show the image tag result in the status like git commit.