search

jpbetz / auger

Directly access data objects stored in etcd by kubernetes.
Apache License 2.0
392 stars 62 forks source link

[WIP] etcd wal log analysis tool etcd-dump-logs support #8

Closed wenjiaswe closed 6 years ago

wenjiaswe commented 6 years ago

This change is a prerequisite to support decoding kubernetes object from the output of etcd wal log analysis tool etcd-dump-logs.

A boolean flag "edl" is added, if set, auger taked output from etcd-dump-logs, which are in the format of [][]byte with '\n' as deliminator. Each []byte need to be decoded and printed to stdout. etcd-dump-logs will then take the stdout from auger and integrated in etcd-dump-logs output.

Here is an example of etcd-dump-logs output using auger as decoder and have kubernetes object decoded:

Snapshot: empty Start dupmping log entries from snapshot. WAL metadata: nodeID=2f7fbfd36802ff7a clusterID=bea638dce5096eef term=2 commitIndex=3842 vote=2f7fbfd36802ff7a WAL entries: lastIndex=3842 term index type data decoded_data decoded_error 1 1 conf method=ConfChangeAddNode id=2f7fbfd36802ff7a <NO DECODED DATA, SEE DECODED_ERROR> error decoding from application/json: Object 'Kind' is missing in '{"id":3422665156528897914,"peerURLs":["https://kubernetes-master:2380"],"name":"etcd-kubernetes-master"}' 2 2 norm <NO DECODED DATA, SEE DECODED_ERROR> error reading input, does not appear to contain valid JSON or binary data 2 3 norm method=PUT path="/0/members/2f7fbfd36802ff7a/attributes" val="{\"name\":\"etcd-kubernetes-master\",\"clientURLs\":[\"http://127.0.0.1:2379\"]}" <NO DECODED DATA, SEE DECODED_ERROR> error reading input, does not appear to contain valid JSON or binary data 2 4 norm method=PUT path="/0/version" val="3.2.0" <NO DECODED DATA, SEE DECODED_ERROR> error reading input, does not appear to contain valid JSON or binary data 2 5 norm header: txn:<compare:<target:MOD key:"/registry/ranges/serviceips" mod_revision:0 > success:<request_put:<key:"/registry/ranges/serviceips" value:"k8s\000\n\025\n\002v1\022\017RangeAllocation\022#\n\022\n\000\022\000\032\000\"\000*\0002\0008\000B\000z\000\022\01310.0.0.0/16\032\000\032\000\"\000" > > failure:<request_range:<key:"/registry/ranges/serviceips" > > > {"kind":"RangeAllocation","apiVersion":"v1","metadata":{"creationTimestamp":null},"range":"10.0.0.0/16","data":""}

Related change on etcd side is here: https://github.com/wenjiaswe/etcd/commit/4ebc794afe3883b1d12ceef261fb41c75749f94c

wenjiaswe commented 6 years ago

I am still working on test cases for both auger and etcd-dump-logs.