ashwanthkumar / gocd-janitor

Attempt at a more intelligent GoCD purge policy
16 stars 11 forks source link

Getting Whitelisted #13

Closed iemkamran closed 6 years ago

iemkamran commented 6 years ago

Hi, All my modules are getting whitelisted so the required work is done is zero. the delete action is not performed so can you please help?

ashwanthkumar commented 6 years ago

Can you please share the configuration that you're using for running the janitor?

iemkamran commented 6 years ago

ashwanthkumar commented 6 years ago

@iemkamran You can use 3 ticks to wrap the configuration

so they look like code and not formatted

iemkamran commented 6 years ago
gocd.janitor {
  server = "http://10.10.2.166:8153"
  username = ""
  password = ""

  # Path to the location where we've all the pipeline directories
  artifacts-dir = "/home/go/go-server/artifacts/pipelines/"

  # Default number of *successful* versions to keep for all the pipelines
   pipeline-versions = 100

  # (Optional) Global prefix while selecting the pipelines (default - "")
  #pipeline-prefix = ""

  # Override the versions to keep for specific pipelines
  # To leave this key blank, specify it like: pipelines = []
  pipelines = []
  # name = "adminportal"
   # runs = 1
 # }]
}
iemkamran commented 6 years ago

what tricks?

ashwanthkumar commented 6 years ago

@iemkamran I edited the comment for you.

I think the problem might be related pipeline-versions = 100. For each pipeline version, we keep all of it's upstream dependencies. For example, consider you've 3 pipelines A, B and C. Also consider the following roles

It's very common for A to run more frequently than B, and B to run more frequently than C. Again if we want to some numbers to explain that,

Pipeline A - 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 (has 10 runs in total) Pipeline B - 1, 2, 3, 4, 5 Pipeline C - 1, 2

Now Run 1 of Pipeline C depends on Run 3 of Pipeline B, which in turn depends on Run 5 of Pipeline A. Similarly C-2 -> B-5 -> A-10, etc.

Now if you say you want to keep latest 2 versions (for example).

  1. For C-1, we'll keep B-3 and A-5.
  2. For pipeline B, we want to keep B-5 and B-4 (latest 2). So in total it's 3 versions of B (that's whitelisted).
  3. For pipeline A, we want to keep 10 and 9 (latest 2) and we'll end up whitelisting other versions if B and C transitively depend on them too.

Above example is just for latest 2 versions. Imagine if the latest versions is 100. Depending on how complex your dependencies between pipelines are, there is a chance that all (as in your case) will be white-listed. Based on my personal experience, having a number like 5 works best. To balance between having enough versions for all pipelines to work without problems and having latest few versions that can be pushed.

iemkamran commented 6 years ago

oh, great explanation, yes the dependency is very complex and versions are way scattered. so I m hoping that some value should work in my case.

Thanks

ashwanthkumar commented 6 years ago

Closing this issue @iemkamran

Please don't hesitate to re-open the issue if needed.