search

jpbetz / auger

Directly access data objects stored in etcd by kubernetes.
Apache License 2.0
392 stars 62 forks source link

etcd wal log analysis tool etcd-dump-logs support #11

Closed wenjiaswe closed 6 years ago

wenjiaswe commented 6 years ago

(Accidentally closed PR #8 , reopen this new one, Joe, PTAL, thanks!)

This change is a prerequisite to support decoding kubernetes object from the output of etcd wal log analysis tool etcd-dump-logs.

A boolean flag "edl" is added, if set, auger taked output from etcd-dump-logs, which are in the format of [][]byte with '\n' as deliminator. Each []byte need to be decoded and printed to stdout. etcd-dump-logs will then take the stdout from auger and integrated in etcd-dump-logs output.

Here is an example of etcd-dump-logs output using auger as decoder and have kubernetes object decoded:

Snapshot: empty Start dupmping log entries from snapshot. WAL metadata: nodeID=2f7fbfd36802ff7a clusterID=bea638dce5096eef term=2 commitIndex=3358 vote=2f7fbfd36802ff7a WAL entries: lastIndex=3358 term index type data decoder_status decoded_data 2 7 norm header: txn:<compare:<target:MOD key:"/registry/ranges/serviceips" mod_revision:0 > success:<request_put:<key:"/registry/ranges/serviceips" value:"k8s\000\n\025\n\002v1\022\017RangeAllocation\022#\n\022\n\000\022\000\032\000\"\000\0002\0008\000B\000z\000\022\01310.0.0.0/16\032\000\032\000\"\000" > > failure:<request_range:<key:"/registry/ranges/serviceips" > > > OK {"kind":"RangeAllocation","apiVersion":"v1","metadata":{"creationTimestamp":null},"range":"10.0.0.0/16","data":""} 2 8 norm header: txn:<compare:<target:MOD key:"/registry/ranges/servicenodeports" mod_revision:0 > success:<request_put:<key:"/registry/ranges/servicenodeports" value:"k8s\000\n\025\n\002v1\022\017RangeAllocation\022#\n\022\n\000\022\000\032\000\"\000\0002\0008\000B\000z\000\022\01330000-32767\032\000\032\000\"\000" > > failure:<request_range:<key:"/registry/ranges/servicenodeports" > > > OK {"kind":"RangeAllocation","apiVersion":"v1","metadata":{"creationTimestamp":null},"range":"30000-32767","data":""} 2 9 norm header: txn:<compare:<target:MOD key:"/registry/apiregistration.k8s.io/apiservices/v1beta1.apps" mod_revision:0 > success:<request_put:<key:"/registry/apiregistration.k8s.io/apiservices/v1beta1.apps" value:"{\"kind\":\"APIService\",\"apiVersion\":\"apiregistration.k8s.io/v1beta1\",\"metadata\":{\"name\":\"v1beta1.apps\",\"uid\":\"f5f8bcad-4d63-11e8-a165-42010a800002\",\"creationTimestamp\":\"2018-05-01T17:20:35Z\",\"labels\":{\"kube-aggregator.kubernetes.io/automanaged\":\"onstart\"}},\"spec\":{\"service\":null,\"group\":\"apps\",\"version\":\"v1beta1\",\"caBundle\":null,\"groupPriorityMinimum\":17800,\"versionPriority\":1},\"status\":{\"conditions\":[{\"type\":\"Available\",\"status\":\"True\",\"lastTransitionTime\":\"2018-05-01T17:20:35Z\",\"reason\":\"Local\",\"message\":\"Local APIServices are always available\"}]}}\n" > > > ERROR:error reading input, does not appear to contain valid JSON or binary data

Related change on etcd side is here: https://github.com/wenjiaswe/etcd/commit/4ebc794afe3883b1d12ceef261fb41c75749f94c

jpbetz commented 6 years ago

@wenjiaswe PRs can be reopened if accidentally closed. Regardless, I've reviewed this replacement PR.