pipeline ballooning the number of builds

[sauloal]

hey guys. we need to talk. there's a serious issue on our pipe-lining. whenever someone creates a push in the container/dev repository, ALL containers are rebuild. this is unacceptable. with ~60 repos, it means that there are 60 builds issued. worse still, the number of different builds to each container also grows so it makes terrible to differentiate between one build and the next (most of the time there will be no difference at all). Added to that, dockerhub makes 2 builds to each request because of the tagging system (program version + latest). so now, at each pull request/push we are creating 120 (40 + 80) new images.

[prvst]

I did some research on this and it seems there is no easy alternative (check here https://forums.docker.com/t/automated-build-only-on-monitored-github-folder-change/257/7).

The team behind Docker Hub is aware of this behavior. Apparently the two only alternatives to this is to move each Dockerfile to a exclusive repository or set the build manually on Docker Hub.

At the moment I believe it is better to maintain the structure as it is.

[bgruening]

At last resort we always can fall back to push images automatically (from travis if needed)? I think keeping the folder structured and maintainable is more important than an easy build setup.

[prvst]

I personally don't think this global build is a big problem for us because we can always use github information to track where updates happened.

On Mon, Oct 5, 2015 at 8:39 AM Björn Grüning notifications@github.com wrote:

At last resort we always can fall back to push images automatically (from travis if needed)? I think keeping the folder structured and maintainable is more important than an easy build setup.

— Reply to this email directly or view it on GitHub https://github.com/BioDocker/biodocker/issues/40#issuecomment-145515341.

[sauloal]

@bgruening @Leprevost the github api is very cumbersome for checking which updates happened.

push images manually is possible but it won't have the 'automatic build' status nor the dockerfile nor the log files in dockerhub. For security reasons I personally never download such packages.

[bgruening]

@sauloal I would also see dockerhub building it for us. But shippable has a nice chain of trust you can encrypt the transfer and so on ... before we stick every container into it's own github repo I would try to use push.

[sauloal]

Shipabble has a big flaw. The dockerfile must be in the root of the repository, therefore, we are not able to have note than one recipe in each repo

On Mon, Oct 5, 2015, 15:25 Björn Grüning notifications@github.com wrote:

@sauloal https://github.com/sauloal I would also see dockerhub building it for us. But shippable has a nice chain of trust you can encrypt the transfer and so on ... before we stick every container into it's own github repo I would try to use push.

— Reply to this email directly or view it on GitHub https://github.com/BioDocker/biodocker/issues/40#issuecomment-145526716.

[bgruening]

Urgs ... we should talk to them :)