search

influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.92k stars 3.55k forks source link

[0.9.5-nightly-827c513] cannot unmarshal array into Go value #4550

Closed sergzin closed 8 years ago

sergzin commented 9 years ago

I am running an influxdb cluster with the following version:

InfluxDB starting, version 0.9.5-nightly-827c513, branch master, commit 827c51384c9b77343cbfc6b3f2e5c1e8bf608bee, built '2015-10-23T04:00:42+0000'
Go version go1.5.1, GOMAXPROCS set to 32

three out of 4 nodes return the following output for show measurements command

# /opt/influxdb/influx -database observer
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 0.9.5-nightly-827c513
InfluxDB shell 0.9.5-nightly-827c513
>
> show measurements
ERR: json: cannot unmarshal array into Go value of type tsdb.MapperOutputJSON
> show tag keys from firewall
ERR: write message type: write tcp 172.31.141.195:33254->172.31.141.196:8088: use of closed network connection
> show tag keys from traffic
ERR: write message type: write tcp 172.31.141.195:33277->172.31.141.196:8088: use of closed network connection

some other queries works fine

> show tag values from firewall with key=name where type='policer'
name: nameTagValues
-------------------
name
AAA-POLICER-MGT-RADIUS
AAA-POLICER-MGT-TACACS
DOMAIN-POLICER-MGT-DOMAIN
> show tag values from firewall with key=type
name: typeTagValues
-------------------
type
counter
policer

I tried to restart all nodes but it didn't help.

otoolep commented 9 years ago

Any ideas @dgnorton ?

dgnorton commented 9 years ago

@sergzin can you post the influxd log?

beckettsean commented 9 years ago

@sergzin can you also post the results of SHOW SERVERS?

otoolep commented 9 years ago

4561 is also a report of this against a 0.9.5 nightly.

priyadarsh commented 9 years ago

Is it safe to assume that the data being written is getting stored properly but only the retrieving part is failing?

sergzin commented 9 years ago

The issue started after I upgraded to 0.9.5-nightly. I wanted to try new aggregation query syntax select time,max(in_bps) ...

Here is my environment: There were 3 servers in the beginning. I added in September 2 more. Then I have to delete one due to failed disk with DROP SERVER command.

> show servers
id      cluster_addr            raft    raft-leader
1       172.31.141.195:8088     true    true
2       172.31.141.197:8088     true    false
3       172.31.141.196:8088     true    false
4       172.31.141.199:8088     false   false

> show shards
name: observer
--------------
id      start_time              end_time                expiry_time             owners
1       2015-06-23T00:00:00Z    2015-06-24T00:00:00Z    2015-08-23T00:00:00Z    3,1,2
2       2015-06-24T00:00:00Z    2015-06-25T00:00:00Z    2015-08-24T00:00:00Z    2,3,1
3       2015-06-25T00:00:00Z    2015-06-26T00:00:00Z    2015-08-25T00:00:00Z    3,1,2
4       2015-06-26T00:00:00Z    2015-06-27T00:00:00Z    2015-08-26T00:00:00Z    1,2,3
5       2015-06-27T00:00:00Z    2015-06-28T00:00:00Z    2015-08-27T00:00:00Z    2,3,1
6       2015-06-28T00:00:00Z    2015-06-29T00:00:00Z    2015-08-28T00:00:00Z    3,1,2
7       2015-06-29T00:00:00Z    2015-06-30T00:00:00Z    2015-08-29T00:00:00Z    1,2,3
8       2015-06-30T00:00:00Z    2015-07-01T00:00:00Z    2015-08-30T00:00:00Z    2,3,1
9       2015-07-01T00:00:00Z    2015-07-02T00:00:00Z    2015-08-31T00:00:00Z    3,1,2
10      2015-07-02T00:00:00Z    2015-07-03T00:00:00Z    2015-09-01T00:00:00Z    1,2,3
11      2015-07-03T00:00:00Z    2015-07-04T00:00:00Z    2015-09-02T00:00:00Z    2,3,1
12      2015-07-04T00:00:00Z    2015-07-05T00:00:00Z    2015-09-03T00:00:00Z    3,1,2
13      2015-07-05T00:00:00Z    2015-07-06T00:00:00Z    2015-09-04T00:00:00Z    1,2,3
14      2015-07-06T00:00:00Z    2015-07-07T00:00:00Z    2015-09-05T00:00:00Z    2,3,1
15      2015-07-07T00:00:00Z    2015-07-08T00:00:00Z    2015-09-06T00:00:00Z    3,1,2
16      2015-07-08T00:00:00Z    2015-07-09T00:00:00Z    2015-09-07T00:00:00Z    1,2,3
17      2015-07-09T00:00:00Z    2015-07-10T00:00:00Z    2015-09-08T00:00:00Z    2,3,1
18      2015-07-10T00:00:00Z    2015-07-11T00:00:00Z    2015-09-09T00:00:00Z    3,1,2
19      2015-07-11T00:00:00Z    2015-07-12T00:00:00Z    2015-09-10T00:00:00Z    1,2,3
20      2015-07-12T00:00:00Z    2015-07-13T00:00:00Z    2015-09-11T00:00:00Z    2,3,1
21      2015-07-13T00:00:00Z    2015-07-14T00:00:00Z    2015-09-12T00:00:00Z    3,1,2
22      2015-07-14T00:00:00Z    2015-07-15T00:00:00Z    2015-09-13T00:00:00Z    1,2,3
23      2015-07-15T00:00:00Z    2015-07-16T00:00:00Z    2015-09-14T00:00:00Z    2,3,1
24      2015-07-16T00:00:00Z    2015-07-17T00:00:00Z    2015-09-15T00:00:00Z    3,1,2
25      2015-07-17T00:00:00Z    2015-07-18T00:00:00Z    2015-09-16T00:00:00Z    1,2,3
26      2015-07-18T00:00:00Z    2015-07-19T00:00:00Z    2015-09-17T00:00:00Z    2,3,1
27      2015-07-19T00:00:00Z    2015-07-20T00:00:00Z    2015-09-18T00:00:00Z    3,1,2
28      2015-07-20T00:00:00Z    2015-07-21T00:00:00Z    2015-09-19T00:00:00Z    1,2,3
29      2015-07-21T00:00:00Z    2015-07-22T00:00:00Z    2015-09-20T00:00:00Z    2,3,1
30      2015-07-22T00:00:00Z    2015-07-23T00:00:00Z    2015-09-21T00:00:00Z    3,1,2
31      2015-07-23T00:00:00Z    2015-07-24T00:00:00Z    2015-09-22T00:00:00Z    1,2,3
32      2015-07-24T00:00:00Z    2015-07-25T00:00:00Z    2015-09-23T00:00:00Z    2,3,1
33      2015-07-25T00:00:00Z    2015-07-26T00:00:00Z    2015-09-24T00:00:00Z    3,1,2
34      2015-07-26T00:00:00Z    2015-07-27T00:00:00Z    2015-09-25T00:00:00Z    3,1,2
35      2015-07-27T00:00:00Z    2015-07-28T00:00:00Z    2015-09-26T00:00:00Z    1,2,3
36      2015-07-28T00:00:00Z    2015-07-29T00:00:00Z    2015-09-27T00:00:00Z    2,3,1
37      2015-07-29T00:00:00Z    2015-07-30T00:00:00Z    2015-09-28T00:00:00Z    3,1,2
38      2015-07-30T00:00:00Z    2015-07-31T00:00:00Z    2015-09-29T00:00:00Z    1,2,3
39      2015-07-31T00:00:00Z    2015-08-01T00:00:00Z    2015-09-30T00:00:00Z    2,3,1
40      2015-08-01T00:00:00Z    2015-08-02T00:00:00Z    2015-10-01T00:00:00Z    3,1,2
41      2015-08-02T00:00:00Z    2015-08-03T00:00:00Z    2015-10-02T00:00:00Z    1,2,3
42      2015-08-03T00:00:00Z    2015-08-04T00:00:00Z    2015-10-03T00:00:00Z    2,3,1
43      2015-08-04T00:00:00Z    2015-08-05T00:00:00Z    2015-10-04T00:00:00Z    3,1,2
44      2015-08-05T00:00:00Z    2015-08-06T00:00:00Z    2015-10-05T00:00:00Z    1,2,3
45      2015-08-06T00:00:00Z    2015-08-07T00:00:00Z    2015-10-06T00:00:00Z    2,3,1
46      2015-08-07T00:00:00Z    2015-08-08T00:00:00Z    2015-10-07T00:00:00Z    3,1,2
47      2015-08-08T00:00:00Z    2015-08-09T00:00:00Z    2015-10-08T00:00:00Z    1,2,3
48      2015-08-09T00:00:00Z    2015-08-10T00:00:00Z    2015-10-09T00:00:00Z    2,3,1
49      2015-08-10T00:00:00Z    2015-08-11T00:00:00Z    2015-10-10T00:00:00Z    3,1,2
50      2015-08-11T00:00:00Z    2015-08-12T00:00:00Z    2015-10-11T00:00:00Z    1,2,3
51      2015-08-12T00:00:00Z    2015-08-13T00:00:00Z    2015-10-12T00:00:00Z    2,3,1
52      2015-08-13T00:00:00Z    2015-08-14T00:00:00Z    2015-10-13T00:00:00Z    3,1,2
53      2015-08-14T00:00:00Z    2015-08-15T00:00:00Z    2015-10-14T00:00:00Z    1,2,3
54      2015-08-15T00:00:00Z    2015-08-16T00:00:00Z    2015-10-15T00:00:00Z    2,3,1
55      2015-08-16T00:00:00Z    2015-08-17T00:00:00Z    2015-10-16T00:00:00Z    3,1,2
56      2015-08-17T00:00:00Z    2015-08-18T00:00:00Z    2015-10-17T00:00:00Z    1,2,3
57      2015-08-18T00:00:00Z    2015-08-19T00:00:00Z    2015-10-18T00:00:00Z    2,3,1
58      2015-08-19T00:00:00Z    2015-08-20T00:00:00Z    2015-10-19T00:00:00Z    3,1,2
59      2015-08-20T00:00:00Z    2015-08-21T00:00:00Z    2015-10-20T00:00:00Z    1,2,3
60      2015-08-21T00:00:00Z    2015-08-22T00:00:00Z    2015-10-21T00:00:00Z    2,3,1
61      2015-08-22T00:00:00Z    2015-08-23T00:00:00Z    2015-10-22T00:00:00Z    3,1,2
62      2015-08-23T00:00:00Z    2015-08-24T00:00:00Z    2015-10-23T00:00:00Z    1,2,3
63      2015-08-24T00:00:00Z    2015-08-25T00:00:00Z    2015-10-24T00:00:00Z    3,1,2
64      2015-08-25T00:00:00Z    2015-08-26T00:00:00Z    2015-10-25T00:00:00Z    2,3,1
65      2015-08-26T00:00:00Z    2015-08-27T00:00:00Z    2015-10-26T00:00:00Z    1,2,3
66      2015-08-27T00:00:00Z    2015-08-28T00:00:00Z    2015-10-27T00:00:00Z    3,1,2
67      2015-08-28T00:00:00Z    2015-08-29T00:00:00Z    2015-10-28T00:00:00Z    2,3,1
68      2015-08-29T00:00:00Z    2015-08-30T00:00:00Z    2015-10-29T00:00:00Z    1,2,3
69      2015-08-30T00:00:00Z    2015-08-31T00:00:00Z    2015-10-30T00:00:00Z    3,1,2
70      2015-08-31T00:00:00Z    2015-09-01T00:00:00Z    2015-10-31T00:00:00Z    2,3,1
71      2015-09-01T00:00:00Z    2015-09-02T00:00:00Z    2015-11-01T00:00:00Z    1,2,3
72      2015-09-02T00:00:00Z    2015-09-03T00:00:00Z    2015-11-02T00:00:00Z    3,1,2
73      2015-09-03T00:00:00Z    2015-09-04T00:00:00Z    2015-11-03T00:00:00Z    2,3,1
74      2015-09-04T00:00:00Z    2015-09-05T00:00:00Z    2015-11-04T00:00:00Z    1,2,3
75      2015-09-05T00:00:00Z    2015-09-06T00:00:00Z    2015-11-05T00:00:00Z    3,1,2
76      2015-09-06T00:00:00Z    2015-09-07T00:00:00Z    2015-11-06T00:00:00Z    2,3,1
77      2015-09-07T00:00:00Z    2015-09-08T00:00:00Z    2015-11-07T00:00:00Z    1,2,3
78      2015-09-08T00:00:00Z    2015-09-09T00:00:00Z    2015-11-08T00:00:00Z    3,1,2
79      2015-09-09T00:00:00Z    2015-09-10T00:00:00Z    2015-11-09T00:00:00Z    2,3,1
80      2015-09-10T00:00:00Z    2015-09-11T00:00:00Z    2015-11-10T00:00:00Z    1,2,3
81      2015-09-11T00:00:00Z    2015-09-12T00:00:00Z    2015-11-11T00:00:00Z    3,1,2
82      2015-09-12T00:00:00Z    2015-09-13T00:00:00Z    2015-11-12T00:00:00Z    2,3,1
83      2015-09-13T00:00:00Z    2015-09-14T00:00:00Z    2015-11-13T00:00:00Z    1,2,3
84      2015-09-14T00:00:00Z    2015-09-15T00:00:00Z    2015-11-14T00:00:00Z    3,1,2
85      2015-09-15T00:00:00Z    2015-09-16T00:00:00Z    2015-11-15T00:00:00Z    2,3,1
86      2015-09-16T00:00:00Z    2015-09-17T00:00:00Z    2015-11-16T00:00:00Z    1,2,3
87      2015-09-17T00:00:00Z    2015-09-18T00:00:00Z    2015-11-17T00:00:00Z    3,1,2
88      2015-09-18T00:00:00Z    2015-09-19T00:00:00Z    2015-11-18T00:00:00Z    2,3,1
89      2015-09-19T00:00:00Z    2015-09-20T00:00:00Z    2015-11-19T00:00:00Z    1,2,3
90      2015-09-20T00:00:00Z    2015-09-21T00:00:00Z    2015-11-20T00:00:00Z    3,1,2
91      2015-09-21T00:00:00Z    2015-09-22T00:00:00Z    2015-11-21T00:00:00Z    2,3,1
92      2015-09-22T00:00:00Z    2015-09-23T00:00:00Z    2015-11-22T00:00:00Z    1,2,3
93      2015-09-23T00:00:00Z    2015-09-24T00:00:00Z    2015-11-23T00:00:00Z    3,1,2
94      2015-09-24T00:00:00Z    2015-09-25T00:00:00Z    2015-11-24T00:00:00Z    2,3,1
95      2015-09-25T00:00:00Z    2015-09-26T00:00:00Z    2015-11-25T00:00:00Z    1,2,3
99      2015-09-26T00:00:00Z    2015-09-27T00:00:00Z    2015-11-26T00:00:00Z    2,3,1
103     2015-09-27T00:00:00Z    2015-09-28T00:00:00Z    2015-11-27T00:00:00Z    2,3,1
107     2015-09-28T00:00:00Z    2015-09-29T00:00:00Z    2015-11-28T00:00:00Z    2,3,1
111     2015-09-29T00:00:00Z    2015-09-30T00:00:00Z    2015-11-29T00:00:00Z    1,2,3
115     2015-09-30T00:00:00Z    2015-10-01T00:00:00Z    2015-11-30T00:00:00Z    1,2,3
119     2015-10-01T00:00:00Z    2015-10-02T00:00:00Z    2015-12-01T00:00:00Z    2,3,1
125     2015-10-02T00:00:00Z    2015-10-03T00:00:00Z    2015-12-02T00:00:00Z    4,1
134     2015-10-03T00:00:00Z    2015-10-04T00:00:00Z    2015-12-03T00:00:00Z    4,1
143     2015-10-04T00:00:00Z    2015-10-05T00:00:00Z    2015-12-04T00:00:00Z    1,2
152     2015-10-05T00:00:00Z    2015-10-06T00:00:00Z    2015-12-05T00:00:00Z    1,2,3
161     2015-10-06T00:00:00Z    2015-10-07T00:00:00Z    2015-12-06T00:00:00Z    1,2,3
168     2015-10-07T00:00:00Z    2015-10-08T00:00:00Z    2015-12-07T00:00:00Z    1,2,3
175     2015-10-08T00:00:00Z    2015-10-09T00:00:00Z    2015-12-08T00:00:00Z    1,2,3
182     2015-10-09T00:00:00Z    2015-10-10T00:00:00Z    2015-12-09T00:00:00Z    1,2,3
189     2015-10-10T00:00:00Z    2015-10-11T00:00:00Z    2015-12-10T00:00:00Z    1,2,3
196     2015-10-11T00:00:00Z    2015-10-12T00:00:00Z    2015-12-11T00:00:00Z    1,2,3
203     2015-10-12T00:00:00Z    2015-10-13T00:00:00Z    2015-12-12T00:00:00Z    1,2,3
210     2015-10-13T00:00:00Z    2015-10-14T00:00:00Z    2015-12-13T00:00:00Z    1,2,3
217     2015-10-14T00:00:00Z    2015-10-15T00:00:00Z    2015-12-14T00:00:00Z    1,2,3
224     2015-10-15T00:00:00Z    2015-10-16T00:00:00Z    2015-12-15T00:00:00Z    1,2,3
231     2015-10-16T00:00:00Z    2015-10-17T00:00:00Z    2015-12-16T00:00:00Z    1,2,3
238     2015-10-17T00:00:00Z    2015-10-18T00:00:00Z    2015-12-17T00:00:00Z    1,2,3
245     2015-10-18T00:00:00Z    2015-10-19T00:00:00Z    2015-12-18T00:00:00Z    2,3,4
252     2015-10-19T00:00:00Z    2015-10-20T00:00:00Z    2015-12-19T00:00:00Z    3,4
260     2015-10-20T00:00:00Z    2015-10-21T00:00:00Z    2015-12-20T00:00:00Z    1,2,3
267     2015-10-21T00:00:00Z    2015-10-22T00:00:00Z    2015-12-21T00:00:00Z    1,2,3
274     2015-10-22T00:00:00Z    2015-10-23T00:00:00Z    2015-12-22T00:00:00Z    2,3,4
281     2015-10-23T00:00:00Z    2015-10-24T00:00:00Z    2015-12-23T00:00:00Z    2,3,4
288     2015-10-24T00:00:00Z    2015-10-25T00:00:00Z    2015-12-24T00:00:00Z    4,1,2
290     2015-10-25T00:00:00Z    2015-10-26T00:00:00Z    2015-12-25T00:00:00Z    2,3,4
292     2015-10-26T00:00:00Z    2015-10-27T00:00:00Z    2015-12-26T00:00:00Z    2,3,4

# cat /etc/opt/influxdb/influxdb.conf
reporting-disabled = true
[registration]
  enabled = false
[meta]
  dir = "/data/influxdb/meta"
  hostname = "172.31.141.195"
  bind-address = ":8088"
  retention-autocreate = true
  election-timeout = "1s"
  heartbeat-timeout = "1s"
  leader-lease-timeout = "500ms"
  commit-timeout = "50ms"
  peers = ["172.31.141.195:8088", "172.31.141.196:8088", "172.31.141.197:8088"]
[data]
  dir = "/data/influxdb/data"
  max-wal-size = 1048576000 # 1G
  wal-flush-interval = "10m" # Maximum time data can sit in WAL before a flush.
  wal-partition-flush-delay = "2s" # The delay time between each WAL partition being flushed.
  wal-dir = "/data/influxdb/wal"
  wal-enable-logging = true
[cluster]
  shard-writer-timeout = "10s" # The time within which a shard must respond to write.
  write-timeout = "5s" # The time within which a write operation must complete on the cluster.
[retention]
  enabled = true
  check-interval = "30m"
[monitor]
  store-enabled = false # Whether to record statistics internally.
  store-database = "_internal" # The destination database for recorded statistics
  store-interval = "10s" # The interval at which to record statistics
[admin]
  enabled = true
  bind-address = ":8083"
  https-enabled = false
  https-certificate = "/etc/ssl/influxdb.pem"
[http]
  enabled = true
  bind-address = ":8086"
  auth-enabled = false
  log-enabled = true
  write-tracing = false
  pprof-enabled = false
  https-enabled = false
  https-certificate = "/etc/ssl/influxdb.pem"
[[graphite]]
  enabled = true
  database = "graphite"
  bind-address = ":2003"
  protocol = "tcp"
  consistency-level = "one"
  name-separator = "."
[collectd]
  enabled = false
[opentsdb]
  enabled = false
[[udp]]
  enabled = false
[continuous_queries]
  log-enabled = true
  enabled = true
  recompute-previous-n = 2
  recompute-no-older-than = "10m"
  compute-runs-per-interval = 10
  compute-no-more-than = "2m"
[hinted-handoff]
  enabled = true
  dir = "/data/influxdb/hh"
  max-size = 1073741824
  max-age = "168h"
  retry-rate-limit = 0
  retry-interval = "1s"
  retry-max-interval = "1m"

The following query works fine on all nodes.

/opt/influxdb/influx -database observer -host 172.31.141.199 -execute "select * from traffic where if_id='130' and time > now() - 1h"
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
name: traffic
-------------
time                    ae_if_id        if_id   in_bps                  out_bps                 phy_if_id       router
1445864739000000000     2985902         130     5.65870442107e+09       6.99972102942e+08       0               7
1445865038000000000     2985902         130     5.6335774686e+09        7.33178417017e+08       0               7
1445865345000000000     2985902         130     5.44026585931e+09       7.30231255531e+08       0               7
1445865643000000000     2985902         130     5.78437244838e+09       7.5191753506e+08        0               7
1445865943000000000     2985902         130     5.82072042477e+09       7.79557631013e+08       0               7
1445866242000000000     2985902         130     5.88209280407e+09       7.6136775414e+08        0               7
1445866547000000000     2985902         130     5.73250201136e+09       7.40687404485e+08       0               7
1445866846000000000     2985902         130     5.88932236193e+09       7.88574859104e+08       0               7
1445867140000000000     2985902         130     6.22608571078e+09       7.7124359298e+08        0               7
1445867447000000000     2985902         130     5.8825954277e+09        7.46991306632e+08       0               7
1445867742000000000     2985902         130     6.1448840163e+09        7.57241877803e+08       0               7
1445868045000000000     2985902         130     5.88382954397e+09       7.20051266851e+08       0               7

If I query for tags I get the following output.

Query to server 172.31.141.196:

> show tag keys from traffic
ERR: write message type: write tcp 172.31.141.196:52740->172.31.141.195:8088: use of closed network connection

InfluxDB log on machine 196:

Oct 26 15:29:35 nl-ams02c-tsd02 influxd[20876]: [query] 2015/10/26 15:29:35 SHOW TAG KEYS FROM "observer"."daily".traffic
Oct 26 15:29:35 nl-ams02c-tsd02 influxd[20876]: [http] 2015/10/26 15:29:35 172.31.141.195 - - [26/Oct/2015:15:29:35 +0100] GET /query?db=observer&epoch=ns&q=show+tag+keys+from+traffic HTTP/1.1 200 137 - InfluxDBShell/0.9.5-nightly-827c513 fb3ef3db-7bed-11e5-a188-000000000000 2.820334ms

Log on machine 195:

Oct 26 15:29:35 nl-ams02c-tsd01 influxd[22065]: [tcp] 2015/10/26 15:29:35 accept remote connection from 172.31.141.196:52740
Oct 26 15:29:35 nl-ams02c-tsd01 influxd[22065]: [tcp] 2015/10/26 15:29:35 accept remote connection from 172.31.141.196:52741
Oct 26 15:29:35 nl-ams02c-tsd01 influxd[22065]: [tcp] 2015/10/26 15:29:35 close remote connection from 172.31.141.196:52741
Oct 26 15:29:35 nl-ams02c-tsd01 influxd[22065]: [tcp] 2015/10/26 15:29:35 close remote connection from 172.31.141.196:52740
sergzin commented 9 years ago

Another might be interesting example from logs on all servers:

InfluxDB CLI:

> show measurements
ERR: json: cannot unmarshal array into Go value of type tsdb.MapperOutputJSON
> show measurements
ERR: json: cannot unmarshal array into Go value of type tsdb.MapperOutputJSON
> show measurements
ERR: json: cannot unmarshal array into Go value of type tsdb.MapperOutputJSON
> show tag keys from firewall
ERR: write message type: write tcp 172.31.141.195:54264->172.31.141.197:8088: use of closed network connection
>

Query log on server .195:

ct 26 16:36:44 nl-ams02c-tsd01 influxd[22065]: [query] 2015/10/26 16:36:44 SHOW MEASUREMENTS
Oct 26 16:36:44 nl-ams02c-tsd01 influxd[22065]: [http] 2015/10/26 16:36:44 172.31.141.195 - - [26/Oct/2015:16:36:44 +0100] GET /query?db=observer&epoch=ns&q=show+measurements HTTP/1.1 200 115 - InfluxDBShell/0.9.5-nightly-827c513 5ca43515-7bf7-11e5-be50-000000000000 7.789705ms

Oct 26 16:36:58 nl-ams02c-tsd01 influxd[22065]: [query] 2015/10/26 16:36:58 SHOW MEASUREMENTS
Oct 26 16:36:58 nl-ams02c-tsd01 influxd[22065]: [http] 2015/10/26 16:36:58 172.31.141.195 - - [26/Oct/2015:16:36:58 +0100] GET /query?db=observer&epoch=ns&q=show+measurements HTTP/1.1 200 115 - InfluxDBShell/0.9.5-nightly-827c513 64e06343-7bf7-11e5-be6f-000000000000 7.947113ms

Oct 26 16:38:12 nl-ams02c-tsd01 influxd[22065]: [query] 2015/10/26 16:38:12 SHOW MEASUREMENTS
Oct 26 16:38:12 nl-ams02c-tsd01 influxd[22065]: [http] 2015/10/26 16:38:12 172.31.141.195 - - [26/Oct/2015:16:38:12 +0100] GET /query?db=observer&epoch=ns&q=show+measurements HTTP/1.1 200 115 - InfluxDBShell/0.9.5-nightly-827c513 9101279f-7bf7-11e5-800c-000000000000 8.273568ms

InfluxDB log on .196:

Oct 26 16:38:26 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:26 accept remote connection from 172.31.141.195:45233
Oct 26 16:38:26 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:26 accept remote connection from 172.31.141.195:45236
Oct 26 16:38:26 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:26 accept remote connection from 172.31.141.195:45237
Oct 26 16:38:26 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:26 close remote connection from 172.31.141.195:45233
Oct 26 16:38:26 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:26 close remote connection from 172.31.141.195:45237
Oct 26 16:38:26 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:26 close remote connection from 172.31.141.195:45236

Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45172: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45172
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45175: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45175
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45171: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45171
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45176: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45176
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45159: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45159
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45217: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45217
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45219: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45219
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45221: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45221
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.196:8088->172.31.141.195:45222: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd02 influxd[20876]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:45222

InfluxDB log on .197:

Oct 26 16:36:34 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:34 accept remote connection from 172.31.141.195:54176
Oct 26 16:36:34 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:34 accept remote connection from 172.31.141.195:54178
Oct 26 16:36:34 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:34 accept remote connection from 172.31.141.195:54180
Oct 26 16:36:34 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:34 close remote connection from 172.31.141.195:54176
Oct 26 16:36:34 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:34 close remote connection from 172.31.141.195:54178
Oct 26 16:36:34 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:34 close remote connection from 172.31.141.195:54180
Oct 26 16:36:44 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:44 accept remote connection from 172.31.141.195:54189
Oct 26 16:36:44 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:36:44 accept remote connection from 172.31.141.195:54192

Oct 26 16:38:12 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:12 accept remote connection from 172.31.141.195:54250
Oct 26 16:38:26 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:26 accept remote connection from 172.31.141.195:54264
Oct 26 16:38:26 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:26 accept remote connection from 172.31.141.195:54267
Oct 26 16:38:26 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:26 close remote connection from 172.31.141.195:54264
Oct 26 16:38:26 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:26 close remote connection from 172.31.141.195:54267
Oct 26 16:38:33 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.197:8088->172.31.141.195:54192: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:54192
Oct 26 16:38:33 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.197:8088->172.31.141.195:54189: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:54189
Oct 26 16:38:33 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.197:8088->172.31.141.195:54250: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd03 influxd[27411]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:54250

InfluxDB log on .199:

Oct 26 16:36:34 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:34 accept remote connection from 172.31.141.195:59441
Oct 26 16:36:34 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:34 close remote connection from 172.31.141.195:59441
Oct 26 16:36:34 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:34 accept remote connection from 172.31.141.195:59445
Oct 26 16:36:34 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:34 close remote connection from 172.31.141.195:59445
Oct 26 16:36:44 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:44 accept remote connection from 172.31.141.195:59452
Oct 26 16:36:44 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:44 accept remote connection from 172.31.141.195:59454
Oct 26 16:36:44 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:44 accept remote connection from 172.31.141.195:59457

Oct 26 16:36:58 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:58 accept remote connection from 172.31.141.195:59470
Oct 26 16:36:58 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:36:58 accept remote connection from 172.31.141.195:59469

Oct 26 16:38:12 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:12 accept remote connection from 172.31.141.195:59516

Oct 26 16:38:26 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:26 accept remote connection from 172.31.141.195:59530
Oct 26 16:38:26 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:26 close remote connection from 172.31.141.195:59530

Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.199:8088->172.31.141.195:59469: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.199:8088->172.31.141.195:59470: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:59470
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:59469
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.199:8088->172.31.141.195:59452: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:59452
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.199:8088->172.31.141.195:59454: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.199:8088->172.31.141.195:59457: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:59457
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:59454
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 unable to read type-length-value read message type: read tcp 172.31.141.199:8088->172.31.141.195:59516: read: connection reset by peer
Oct 26 16:38:33 nl-ams02c-tsd05 influxd[43186]: [tcp] 2015/10/26 16:38:33 close remote connection from 172.31.141.195:59516

After SHOW MEASUREMENTS commands I see only accept message in log. There are no close message.

sergzin commented 9 years ago

I see similar issues with test database if replication factor less than number of servers in cluster.

li-ang commented 8 years ago

@dgnorton I just fix the SHOW MEASUREMENTS. If the PR #4833 could be merged, I would like to try to fix SHOW TAG KEYS and SHOW TAG VALUES using the similar method.

beckettsean commented 8 years ago

fixed by https://github.com/influxdata/influxdb/pull/4833