search

CentOS-PaaS-SIG / contra-env-setup

This repo is for setting up the continuous-infra pipeline environment.
GNU General Public License v3.0
12 stars 29 forks source link

Multiple resource creation from template is not idempotent #115

Closed ifireball closed 5 years ago

ifireball commented 5 years ago

With the current code resources are created from templates only if:

  1. The image stream created by the template does not exist
  2. The build config created by the template dos not exist
  3. The container created by the template was never built

As a result, the following example situations can occur:

  1. If the image stream exists and the build config does not - it will not get created
  2. If the template defines a deployment config, it will not get created if an image stream or a build already exists
arilivigni commented 5 years ago

do you have proof of that? This is a feature feel free to create a PR if you desire that functionality. We welcome contributions.

ifireball commented 5 years ago

do you have proof of that?

Yep, reproduced it quite a few times, its rather easy to reproduce:

  1. Run the playbook and have everything installed
  2. Go to the cluster erase for e.g. the jenkins deployment config
  3. Rerun the playbook - the deployment config will not reappear.

This is a feature

AFAIK when using Ansbile one expects things to mostly be idempotent. So some may regard this a s a bug.

arilivigni commented 5 years ago

Point taken not at the top of our list feel free to file a PR if we don't get to it first.

arilivigni commented 5 years ago

In thinking about this further I have a difference of opinion. This is calling ansible playbooks but there is no context of projects and configs inside OpenShift. So in this case we shouldn't be idempotent. This is why we have the run_cleanup option.

  1. How do we know what configs inside the project came from vs what is being run with contra-env-setup?
  2. We could potentially introduce a config that doesn't match an image stream?

The way this was designed was to setup a project what you are proposing changes the intent of the contra-env-setup so I would consider this a design change of what our target was.

ifireball commented 5 years ago

Well, I suppose if everything works for you on the 1st try then you don't need idempotence...

Is there no aspect of this that is needed when maintaining system over time?

arilivigni commented 5 years ago

I don't believe this is accurate if during a run the build or image fails it retried a certain number of times and then is marked as a failure for the run. If this is not the case then there in fact is a bug, but I believe with the cleanup option that it would rerun. We are also ignorant to know if the project tenant we are running on is large enough to support a given project template deployment.

The idea of the implementation was to deploy s2i templates to an OpenShift endpoint and minishift if needed. The s2i templates cover dockerfiles, jenkins pipelines, and native s2i support. I am fine with expanding the scope if it is needed.

Please right up your requirements in detail of what you want the contra-env-setup to cover in a separate issue. Also do you have cycles to assist or are you expecting us to implement it all?

arilivigni commented 5 years ago

I did a lot of testing with my recent change. If a template was not built and I reran it then it got created. If there is a failed build all failed builds get cleaned up so I believe this to be idempotent from run to run.

I will leave this open for a couple of weeks if you can provide an example that does not perform this way. Then I will close this issue.

ifireball commented 5 years ago

The issue it what happens if you got multiple resources in the same template - such as when you have both a build and a deployment config. In that case if the build is in place and the deployment config is not - it will not get created.

Anyway, I see you've made some updates, I'll try to find the time to retry my tests with them.