myENA / consul-backinator

Command line Consul backup and restore utility supporting KVs, ACLs and Queries
Mozilla Public License 2.0
226 stars 22 forks source link

debugging backup? #14

Closed vijaybandari closed 7 years ago

vijaybandari commented 7 years ago

I can see data in my consul through http api but this tool returns with 0 keys. How do I debug the issue?

// 20161007120442
// http://171.10.50.4:8500/v1/kv/?keys&token=access_token
[
  "test",
  "test1",
  "test2"
]

consul-backinator backup -file consul.bak -token access_token -addr 171.10.50.4:8500
2016/10/07 12:07:14 [Success] Backed up 0 keys from 171.10.50.4:8500/ to consul.bak
Keep your backup and signature files in a safe place.
You will need both to restore your data.
aaronhurt commented 7 years ago

I haven't seen that issue before. Let me setup some test cases here locally and see if I can reproduce the issue.

aaronhurt commented 7 years ago

@vijaybandari could you please provide your consul agent configuration?

vijaybandari commented 7 years ago

@leprechau Here is my consul configuration:

[root@px-consul-01-rhel-71 0.7.0]# cat /etc/consul/consul.json
{
  "acl_datacenter": "px",
  "acl_default_policy": "deny",
  "advertise_addr": "171.10.50.4",
  "advertise_addr_wan": "171.10.50.4",
  "bind_addr": "171.10.50.4",
  "bootstrap_expect": 3,
  "client_addr": "0.0.0.0",
  "data_dir": "/var/lib/consul",
  "datacenter": "px",
  "ports": {
    "dns": 8600,
    "http": 8500,
    "rpc": 8400,
    "serf_lan": 8301,
    "serf_wan": 8302,
    "server": 8300,
    "https": 9000
  },
  "retry_join": [
    "171.10.50.4",
    "171.10.50.5",
    "171.10.50.6"
  ],
  "server": true,
  "start_join": [
    "171.10.50.4",
    "171.10.50.5",
    "171.10.50.6"
  ],
  "ui": true,
  "ui_dir": "/srv/consul-ui",
  "verify_incoming": false,
  "verify_outgoing": false
}[root@px-consul-01-rhel-71 0.7.0]#
aaronhurt commented 7 years ago

Could you also see what a call here returns ...

http://171.10.50.4:8500/v1/kv/?recurse&token=access_token

You should see something like this ...

[
  {
    "LockIndex": 0,
    "Key": "testKey",
    "Flags": 0,
    "Value": "dGVzdEtleSB2YWx1ZQ==",
    "CreateIndex": 58,
    "ModifyIndex": 58
  },
  {
    "LockIndex": 0,
    "Key": "testKey2",
    "Flags": 0,
    "Value": "dGVzdEtleSB2YWx1ZTI=",
    "CreateIndex": 63,
    "ModifyIndex": 63
  }
]
vijaybandari commented 7 years ago

@leprechau I didn't get you. Something missing in your previous comment?

aaronhurt commented 7 years ago

I think I see the issue ... the backup command doesn't seem to be passing the token value to the recursive kv call.

    2016/10/07 15:07:14 [DEBUG] http: Request GET /v1/kv/?consistent=&recurse= (115.846µs) from=127.0.0.1:56160
    2016/10/07 15:07:31 [DEBUG] http: Request GET /v1/kv/?consistent=&recurse= (103.038µs) from=127.0.0.1:56172
aaronhurt commented 7 years ago

@vijaybandari This appears to be a bug in the upstream consul/api package.

I've filed https://github.com/hashicorp/consul/issues/2403 ... I'll update this as soon as I hear back from upstream.

aaronhurt commented 7 years ago

I've filed https://github.com/hashicorp/consul/issues/2403 ... I'll update this as soon as I hear back from the upstream package.

vijaybandari commented 7 years ago

Thanks @leprechau

aaronhurt commented 7 years ago

Please test the latest master if you can. I'll publish a new release if this resolves your issue. I've rolled back the upstream API to the 0.6.4 version.

vijaybandari commented 7 years ago

Thank @leprechau quick response. v 1.3 works fine 👍