Unexpected EOF - Githubissues

gsmethells commented 6 years ago

Snap daemon version (use snapteld -v): snapteld version 2.0.0 Environment:

Cloud provider or hardware configuration: private cloud using an OpenStack VM
OS (e.g. from /etc/os-release): CentOS release 6.7 (Final)
Kernel (e.g. uname -a): Linux jointsprod-haproxy-new 2.6.32-573.18.1.el6.x86_64 #1 SMP Tue Feb 9 22:46:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Relevant tools (e.g. plugins used with Snap): Used under AppOptics (https://www.appoptics.com)
Others (e.g. deploying with Ansible):

What happened:

Got an "unexpected EOF"

[root@jointsprod-haproxy-new etc]# pwd
/opt/appoptics/etc
[root@jointsprod-haproxy-new etc]# cat plugins.d/haproxy.yaml 
collector:
  haproxy:
    all:
      socket: "/var/run/haproxy.sock"
load:
  plugin: snap-plugin-collector-haproxy
  task: task-haproxy.yaml
[root@jointsprod-haproxy-new etc]# cat tasks.d/task-haproxy.yaml 
---
version: 1
schedule:
  type: "simple"
  interval: "1s"
workflow:
  collect:
    metrics:
      /intel/haproxy/info/ConnRate : {}
    publish:
      - plugin_name: publisher-appoptics
        config:
          period: 60
          floor_seconds: 60
[root@jointsprod-haproxy-new etc]# service appoptics-snapteld restart
* Restarting AppOptics Host Agent Service ...
* AppOptics Host Agent Service has restarted.              [  OK  ]
[root@jointsprod-haproxy-new etc]# snaptel plugin list
NAME             VERSION     TYPE        SIGNED      STATUS      LOADED TIME
aosystem         36          collector   false       loaded      Fri, 17 Aug 2018 12:11:27 CDT
publisher-appoptics      33          publisher   false       loaded      Fri, 17 Aug 2018 12:11:27 CDT
[root@jointsprod-haproxy-new etc]# tail -10 /var/log/appoptics/snapteld.log
ERRO[2018-08-17T12:11:27-05:00] unable to kill client                         error=rpc error: code = Unavailable desc = transport is closing reason=Retrieved necessary plugin info. Send kill command to allow plugin to free resources and exit.
ERRO[2018-08-17T12:11:27-05:00] load plugin error while trying to send kill message to plugin executable  _block=load-plugin _module=control-plugin-mgr error=rpc error: code = Unavailable desc = transport is closing
WARN[2018-08-17T12:11:28-05:00] This plugin is using a deprecated RPC protocol. Find more information here: https://github.com/intelsdi-x/snap/issues/1289   _block=newAvailablePlugin _module=control-aplugin plugin_name=haproxy
ERRO[2018-08-17T12:11:28-05:00] error in getting metric types                 _block=load-plugin _module=control-plugin-mgr error=unexpected EOF plugin-name=haproxy plugin-type=collector plugin-version=4
ERRO[2018-08-17T12:11:28-05:00] unexpected EOF                                _block=autoDiscoverPlugins _module=control plugin=/opt/appoptics/bin/snap-plugin-collector-haproxy
ERRO[2018-08-17T12:11:28-05:00] error getting metric                          _block=get-metrics _file=metrics.go, _module=control error=Metric not found: /intel/haproxy/info/ConnRate (version: 0)
ERRO[2018-08-17T12:11:28-05:00] error matching requested namespace with metric catalog  _block=control action=expanding-requested-metrics err=Metric not found: /intel/haproxy/info/ConnRate (version: 0) query=/intel/haproxy/info/ConnRate
ERRO[2018-08-17T12:11:28-05:00] error creating task                           _error=Metric not found: /intel/haproxy/info/ConnRate (version: 0) _fields=map[] _file=core/task.go _function=CreateTaskFromContent
ERRO[2018-08-17T12:11:28-05:00] Metric not found: /intel/haproxy/info/ConnRate (version: 0)  _block=autoDiscoverTasks _module=scheduler autodiscoverpath=/opt/appoptics/etc/tasks.d/task-haproxy.yaml task=/opt/appoptics/etc/tasks.d/task-haproxy.yaml
ERRO[2018-08-17T12:11:28-05:00] time="2018-08-17T12:11:28-05:00" level=error msg="Error getting host information" _module=appoptics block=Publish error="Host information not gathered yet"   _module=plugin-exec io=stderr plugin=publisher-appoptics
[root@jointsprod-haproxy-new etc]#

What you expected to happen:

The plugin to load successfully and report metrics to

Steps to reproduce it (as minimally and precisely as possible):

yum install appoptics-snaptel
curl -L -O https://github.com/intelsdi-x/snap-plugin-collector-haproxy/releases/download/4/snap-plugin-collector-haproxy_linux_x86_64
mv snap-plugin-collector-haproxy_linux_x86_64 /opt/appoptics/bin/snap-plugin-collector-haproxy
chmod 775 /opt/appoptics/bin/snap-plugin-collector-haproxy
chown appoptics:appoptics /opt/appoptics/bin/snap-plugin-collector-haproxy

vim /opt/appoptics/etc/plugins.d/haproxy.yaml

collector:
haproxy:
all:
  socket: "/var/run/haproxy.sock"
load:
plugin: snap-plugin-collector-haproxy
task: task-haproxy.yaml

vim /opt/appoptics/etc/tasks.d/task-haproxy.yaml

---
version: 1
schedule:
type: "simple"
interval: "1s"
workflow:
collect:
metrics:
  /intel/haproxy/info/ConnRate : {}
publish:
  - plugin_name: publisher-appoptics
    config:
      period: 60
      floor_seconds: 60

yum install haproxy

vim /etc/haproxy/haproxy.cfg


global
maxconn 10000
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 2m

... lots more lines ...

10. `service haproxy restart`
11. `service appoptics-snapteld restart`
12. See the "Unexpected EOF" error on the console and in the logs:

Error loading plugin: unexpected EOF

and

[root@jointsprod-haproxy-new me1]# tail -10 /var/log/appoptics/snapteld.log ERRO[2018-08-17T16:31:38-05:00] unable to kill client error=rpc error: code = Unavailable desc = transport is closing reason=Retrieved necessary plugin info. Send kill command to allow plugin to free resources and exit. ERRO[2018-08-17T16:31:38-05:00] load plugin error while trying to send kill message to plugin executable _block=load-plugin _module=control-plugin-mgr error=rpc error: code = Unavailable desc = transport is closing WARN[2018-08-17T16:31:39-05:00] This plugin is using a deprecated RPC protocol. Find more information here: https://github.com/intelsdi-x/snap/issues/1289 _block=newAvailablePlugin _module=control-aplugin plugin_name=haproxy ERRO[2018-08-17T16:31:39-05:00] error in getting metric types _block=load-plugin _module=control-plugin-mgr error=unexpected EOF plugin-name=haproxy plugin-type=collector plugin-version=4 ERRO[2018-08-17T16:31:39-05:00] unexpected EOF _block=autoDiscoverPlugins _module=control plugin=/opt/appoptics/bin/snap-plugin-collector-haproxy ERRO[2018-08-17T16:31:40-05:00] error getting metric _block=get-metrics _file=metrics.go, _module=control error=Metric not found: /intel/haproxy/info/ConnRate (version: 0) ERRO[2018-08-17T16:31:40-05:00] error matching requested namespace with metric catalog _block=control action=expanding-requested-metrics err=Metric not found: /intel/haproxy/info/ConnRate (version: 0) query=/intel/haproxy/info/ConnRate ERRO[2018-08-17T16:31:40-05:00] error creating task _error=Metric not found: /intel/haproxy/info/ConnRate (version: 0) _fields=map[] _file=core/task.go _function=CreateTaskFromContent ERRO[2018-08-17T16:31:40-05:00] Metric not found: /intel/haproxy/info/ConnRate (version: 0) _block=autoDiscoverTasks _module=scheduler autodiscoverpath=/opt/appoptics/etc/tasks.d/task-haproxy.yaml task=/opt/appoptics/etc/tasks.d/task-haproxy.yaml ERRO[2018-08-17T16:31:40-05:00] time="2018-08-17T16:31:40-05:00" level=error msg="Error getting host information" _module=appoptics block=Publish error="Host information not gathered yet" _module=plugin-exec io=stderr plugin=publisher-appoptics



**Anything else do we need to know** (e.g. issue happens only occasionally):

AppOptics does not "officially" support your plugin. This would be wonderful if they could, because we use haproxy and not varnish because we use SSL/TLS.

gsmethells commented 6 years ago

Is this plugin still supported??

donzh commented 6 years ago

@gsmethells It seems that the issue is caused by the following Error message. ERRO[2018-08-17T12:11:28-05:00] error getting metric _block=get-metrics _file=metrics.go, _module=control error=Metric not found: /intel/haproxy/info/ConnRate (version: 0)

// GetMetrics returns all MTs at the given namespace in the queried version (or in the latest if ver < 1) // and does gather all the node's descendants if the namespace ends with an asterisk func (mtt mttNode) GetMetrics(ns []string, ver int) ([]metricType, error) { nodes := []mttNode{} mts := []metricType{}

if len(ns) == 0 { return nil, errorEmptyNamespace() } // search returns all of the nodes fulfilling the 'ns' // even for some of them there is no metric (empty node.mts) nodes = mtt.search(nodes, ns)

for _, node := range nodes { // choose the queried version of metric types (or the latest if ver < 1) // and concatenate them into a single slice mt, err := getVersion(node.mts, ver) if err != nil { continue } mts = append(mts, mt) } if len(mts) == 0 { return nil, errorMetricNotFound("/"+strings.Join(ns, "/"), ver) //Here is the Error Message! } return mts, nil }

// getVersion returns the MT in the queried version (or the latest if 'ver' < 1) func getVersion(mts map[int]metricType, ver int) (metricType, error) { if len(mts) == 0 { return nil, errMetricNotFound } if ver > 0 { // a version IS given if mt, exist := mts[ver]; exist { return mt, nil } return nil, errMetricNotFound } // or get the latest return getLatest(mts), nil }

gsmethells commented 6 years ago

@donzh alright, so what can be done about it? Is there a next step?

gsmethells commented 6 years ago

@donzh any update?

intelsdi-x / snap-plugin-collector-haproxy

Unexpected EOF #31