Open Dominik-K opened 7 years ago
Yeah, thats an unfortunate side effect of https://github.com/weaveworks/kubediff/pull/31.
@awh can we improve this?
We can, by teaching kubediff to sort certain lists e.g. .spec.template.spec.containers
should be sorted by .name
before comparison.
is there an update on this? or a workaround?
Getting the same issue. An update would be awesome. Thanks for kubediff
!
I've coded a workaround for this. It's far from perfect because tests are broken and all but that fits my needs. Looks interesting to share anyway.
My understanding is that after the change https://github.com/weaveworks/kubediff/pull/31 every change in any deep after a list results in 'element[X] missing' issue we are talking about. Hence if you change the image of a container (the most likely on a daily change), you have this issue, making the tool quite useless imho. The change make sense in a way that if you remove or add element in a list from running or wished configuration, differences between elements get mixed up and you end up with an horrible diff output e.g.: you remove wished var1 then running var1 is diffed with wished var2, running var2 is diffed with wished var3 and so on... Luckily changes not happening much except for environment variable list.
I then did the following change. Reverted the old behavior for list having same size, and if not return 'element missing' except for .spec.template.spec.containers[X].env whish have a special treat.
Let's say you running on this
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
namespace: test
spec:
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.1
ports:
- containerPort: 80
name: http
protocol: TCP
env:
- name: name1
value: value1
- name: name2
value: value2
- name: name3
value: value3
and want this:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx
namespace: test
spec:
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
name: http
protocol: TCP
env:
- name: name1
value: value1.1
- name: name4
value: value4
With my hack, kubediff output is:
## test/nginx (Deployment.v1beta1.extensions)
.spec.template.spec.containers[0].image: 'nginx:1.14.2' != 'nginx:1.14.1'
## test/nginx (Deployment.v1beta1.extensions)
.spec.template.spec.containers[0].env: Unequal lengths: 2 != 3
## test/nginx (Deployment.v1beta1.extensions)
.spec.template.spec.containers[0].env: Diff: name4 in wished config only
## test/nginx (Deployment.v1beta1.extensions)
.spec.template.spec.containers[0].env: Diff: name2 in running config only
## test/nginx (Deployment.v1beta1.extensions)
.spec.template.spec.containers[0].env: Diff: name3 in running config only
## test/nginx (Deployment.v1beta1.extensions)
.spec.template.spec.containers[0].env[0].value: 'value1.1' != 'value1'
Patch is attached. Edit kubedifflib/_diff.py and build container with build instructions in doc except "make .uptodate" instead of "make" to bypass tests. patch.txt
Any update here? I have to imagine comparing changes to containers is pretty crucial to most people's use cases.
Trying kubediff for the first time and I get this on an array inside one of the values for my HelmRelease.
Eg:
kubediff consul grafana loki prometheus-k8s vault
## loki/loki (HelmRelease.v1beta1.flux.weave.works)
.spec.values.loki.config.schema_config.configs: 'element [0]' missing
With the following HelmRelease:
apiVersion: flux.weave.works/v1beta1
kind: HelmRelease
metadata:
name: loki
namespace: loki
annotations:
fluxcd.io/automated: "false"
spec:
releaseName: loki
chart:
repository: https://grafana.github.io/loki/charts
name: loki-stack
version: 0.23.0
values:
loki:
enabled: true
service:
type: NodePort
port: 3100
nodePort: 32004
serviceMonitor:
enabled: true
replicas: 3
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: my-aws-secret
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: my-aws-secret
key: AWS_SECRET_ACCESS_KEY
# Consul runs the agent as a daemonset on a host port, so use the downward
# API to get the hostIP
- name: HOST_IP
valueFrom:
fieldRef:
fieldPath: status.hostIP
extraArgs:
# Use extraArgs to set the -consul.hostname configuration using an
# environment variable from the downward API.
"consul.hostname": "$(HOST_IP):8500"
config:
auth_enabled: false
server:
http_listen_port: 3100
http_server_idle_timeout: "30s"
graceful_shutdown_timeout: "30s"
# 64MB = 67108864 bytes
grpc_server_max_send_msg_size: 67108864
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: "168h"
ingester:
chunk_idle_period: 15m
# 256KB
chunk_block_size: 262144
# max attempts to transfer in-memory data to another ingester replica
# before deciding to flush to storage.
# max_transfer_retries: 60
lifecycler:
ring:
replication_factor: 3
heartbeart_timeout: 1m
kvstore:
store: consul
prefix: "loki-collectors/"
consul:
# "host" is unset in this configuration file because it's set
# using extraArgs due to the value coming from an environment
# variable.
httpclienttimeout: "20s"
consistentreads: true
# num_tokens: 512
join_after: "30s"
ingester_client:
grpc_client_config:
# 64MB = 67108864 bytes
max_recv_msg_size: 67108864
storage_config:
aws:
s3: s3://us-west-1/czibolski-rigetti-test
dynamodbconfig:
dynamodb: dynamodb://us-west-1
schema_config:
configs:
- from: 2019-12-01
store: aws
object_store: s3
schema: v9
index:
prefix: czibolski_loki_test_index_
period: 168h
chunk_store_config:
max_look_back_period: "30s"
table_manager:
retention_deletes_enabled: true
retention_period: 336h
promtail:
enabled: true
Here's the resource in the cluster
k get hr -n loki loki -o yaml
apiVersion: flux.weave.works/v1beta1
kind: HelmRelease
metadata:
annotations:
fluxcd.io/automated: "false"
fluxcd.io/sync-checksum: 34e416b2146d9562dba3c0aab372f593c96a2515
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"flux.weave.works/v1beta1","kind":"HelmRelease","metadata":{"annotations":{"fluxcd.io/automated":"false","fluxcd.io/sync-checksum":"34e416b2146d9562dba3c0aab372f593c96a2515"},"labels":{"fluxc
creationTimestamp: "2019-12-13T17:44:36Z"
generation: 1
labels:
fluxcd.io/sync-gc-mark: sha256.YI3vseFaMevalw7yAnP6WIQ8a5jp58PUacjQdk47ZRQ
name: loki
namespace: loki
resourceVersion: "49026"
selfLink: /apis/flux.weave.works/v1beta1/namespaces/loki/helmreleases/loki
uid: a6c02724-4b6d-4ca5-a306-f8cdffc96b18
spec:
chart:
name: loki-stack
repository: https://grafana.github.io/loki/charts
version: 0.23.0
releaseName: loki
values:
loki:
config:
chart:
name: loki-stack
repository: https://grafana.github.io/loki/charts
version: 0.23.0
releaseName: loki
values:
loki:
config:
auth_enabled: false
chunk_store_config:
max_look_back_period: 30s
ingester:
chunk_block_size: 262144
chunk_idle_period: 15m
lifecycler:
join_after: 30s
ring:
heartbeart_timeout: 1m
kvstore:
consul:
consistentreads: true
httpclienttimeout: 20s
prefix: loki-collectors/
store: consul
replication_factor: 3
ingester_client:
grpc_client_config:
max_recv_msg_size: 67108864
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h
schema_config:
configs:
- from: "2019-12-01"
index:
period: 168h
prefix: czibolski_loki_test_index_
object_store: s3
schema: v9
store: aws
server:
graceful_shutdown_timeout: 30s
grpc_server_max_send_msg_size: 67108864
http_listen_port: 3100
http_server_idle_timeout: 30s
storage_config:
aws:
dynamodbconfig:
dynamodb: dynamodb://us-west-1
The patch from @damienleger works great. thanks
@damienleger I have downloaded the kubediff repo and using the ./kubediff
I am trying to find the differences in k8s state and manifest files. I have modified the _diff.py
file by removing the lines and adding new lines.
Even after that I am still getting the similar issues.
As per your comment "build container with build instructions in doc except "make .uptodate" instead of "make" to bypass tests."
Could you please share the link of the document which you are referring?
hello @gouravsw, tbh I've did that tweak for my previous job so I'm not working on this anymore. That being said, when referring to "doc" I meant this --> https://github.com/weaveworks/kubediff#build
So by replacing make
by make .uptodate
, here it is:
mkdir -p $GOPATH/src/github.com/prometheus && cd "$_"
git clone git@github.com:prometheus/client_golang.git
mkdir -p $GOPATH/src/github.com/weaveworks && cd "$_"
git clone git@github.com:weaveworks/kubediff.git
cd kubediff
make .uptodate
Using this
deployment.yaml
:If I change a field in the
containers
part, e.g. thecontainerPort
,kubediff
returns: