search

intelsdi-x / snap-plugin-collector-elasticsearch

Collects Elasticsearch cluster and nodes statistics
http://snap-telemetry.io/
Apache License 2.0
3 stars 14 forks source link

panic during initialization #7

Closed woodsaj closed 8 years ago

woodsaj commented 8 years ago 
  * /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch_integration_test.go 
  Line 39: - reflect: call of reflect.Value.NumField on zero Value 
  goroutine 6 [running]:
  panic(0x87dbe0, 0xc820165fe0)
    /usr/local/go/src/runtime/panic.go:426 +0x4e9
  reflect.flag.mustBe(0x0, 0x19)
    /usr/local/go/src/reflect/value.go:202 +0xa6
  reflect.Value.NumField(0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/reflect/value.go:1153 +0x2a
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.(*ESMetric).parseData(0xc82011b4a0, 0x7acc40, 0xc82015a3b0, 0x196, 0xc82011b338, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/nodeClient.go:128 +0x562

This was tracked down to the "classes" section missing for some nodes in the response from /_nodes. Is it possible that "classes" is only filled in for the queried node and not other nodes in the cluster?

In any event, missing sections should probably be handled more gracefully then with a panic.

candysmurf commented 8 years ago

@woodsaj , thanks for reporting this. May I know steps to reproduce this error? thanks.

woodsaj commented 8 years ago

1) bring up a ES cluster (at least 2 nodes). 

docker run -d  -p 9200:9200 --net="host" elasticsearch
docker run -d  -p 9201:9200 --net="host" elasticsearch

2) configure the global config

      elasticsearch:
        all:
          server: 127.0.0.1
          port: 9200

3) start snap or change the IP in elasticsearch_integration_test.go to 127.0.0.1 and run the test

Here is the response from http://localhost:9200/_nodes https://gist.github.com/woodsaj/122b7a98ab3839499d9d8459392aafb7

candysmurf commented 8 years ago

woodsaj , By removing --net="host", you should be able to run integration tests. For the integration test, you need to export SNAP_ES_HOST=(your docker container ip).

Please try it and let me know if it works for you. thanks.

woodsaj commented 8 years ago

I have already been testing with the integration test. The test passes because it diesnt check the validity of the metrics returned, it only checks the the count.

But if you iterate over the metrics you will see that they are all empty, ie the zero type of plugin.MetricType

candysmurf commented 8 years ago

@woodsaj. ah. recently, we had metric schema changes. I'll look into it and update you later. thanks.

candysmurf commented 8 years ago

@woodsaj, added more integration tests to check for the actual data in the PR. Please review. thanks.

woodsaj commented 8 years ago

@candysmurf this is still not resolved.

with elasticsearch_integration_test.go updated to point to an ES cluster with a node on 127.0.0.1 i still get a panic.

As mentioned in https://github.com/intelsdi-x/snap-plugin-collector-elasticsearch/issues/7#issuecomment-227642695 you need to run two instances of elasticsearch and the --net="host" option is required to allow the nodes to discover each other and form a cluster.

anthony:~/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch$ go test -run TestESCollectMetrics
E
Errors:

  * /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch_integration_test.go 
  Line 36: - reflect: call of reflect.Value.NumField on zero Value 
  goroutine 6 [running]:
  panic(0x889d20, 0xc820169d20)
    /usr/local/go/src/runtime/panic.go:426 +0x4e9
  reflect.flag.mustBe(0x0, 0x19)
    /usr/local/go/src/reflect/value.go:202 +0xa6
  reflect.Value.NumField(0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/reflect/value.go:1153 +0x2a
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.(*ESMetric).parseData(0xc82011d480, 0x7b7ce0, 0xc82016c370, 0x196, 0xc82011d318, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/nodeClient.go:128 +0x562
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.(*ESMetric).parseData(0xc82011d480, 0x7b8340, 0xc82018c040, 0x196, 0xc82011d318, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/nodeClient.go:144 +0x9fd
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.(*ESMetric).parseData(0xc82011d480, 0x7b84c0, 0xc82018c000, 0x16, 0xc82011d318, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/nodeClient.go:144 +0x9fd
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.(*ESMetric).GetNodeData(0xc82011d480, 0x0, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/nodeClient.go:106 +0xd03
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.getNodeMetrics(0x90eae0, 0xc82000fe00, 0x0, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch.go:131 +0x258
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.getMetrics(0xc82000fe00, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch.go:151 +0x7d
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.(*Elasticsearch).GetMetricTypes(0xbf3160, 0xc82000fe00, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch.go:115 +0x4b
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.TestESCollectMetrics.func1()
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch_integration_test.go:36 +0x58
  github.com/jtolds/gls._m(0x0, 0xc8200cd6a0)
    /home/anthony/go/src/github.com/jtolds/gls/stack_tags.go:39 +0x2b
  github.com/jtolds/gls.markS(0x0, 0xc8200cd6a0)
    /home/anthony/go/src/github.com/jtolds/gls/stack_tags.go:16 +0x2b
  github.com/jtolds/gls.addStackTag(0x0, 0xc8200cd6a0)
    /home/anthony/go/src/github.com/jtolds/gls/stack_tags.go:13 +0x37
  github.com/jtolds/gls.(*ContextManager).SetValues(0xc8200cd5e0, 0xc820118930, 0xc8200cd6a0)
    /home/anthony/go/src/github.com/jtolds/gls/context.go:92 +0x4b3
  github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch.TestESCollectMetrics(0xc820080510)
    /home/anthony/go/src/github.com/intelsdi-x/snap-plugin-collector-elasticsearch/elasticsearch/elasticsearch_integration_test.go:97 +0x1dd
  testing.tRunner(0xc820080510, 0xbc9f60)
    /usr/local/go/src/testing/testing.go:473 +0x98
  created by testing.RunTests
    /usr/local/go/src/testing/testing.go:582 +0x892
candysmurf commented 8 years ago

@woodsaj, if --net="host" has to be there, I'll look into the panic to fix this one. thank you.

candysmurf commented 8 years ago

@woodsaj, A new PR is pushed to address this issue. Please give it a try and let me know if you see any issue. thank you.

candysmurf commented 8 years ago

@woodsaj , PR is merged. Please feel free to test and let me know if any issue. thanks.

woodsaj commented 8 years ago

this is now working. thanks.