search

canonical / layer-filebeat

Filebeat is a lightweight log shipper. This is the source for the filebeat charm in the Juju charm store.
Other
3 stars 27 forks source link

filebeat failed to connect to elasticsearch #45

Closed AshleyLai closed 6 years ago

AshleyLai commented 6 years ago

We have a few runs with filebeat failed connecting to elasticsearch. In the crashdump, juju status shows elasticsearch is still executing. We got an error from juju wait showing filebeat/1 failed.

foundation/bin/log_wrapper juju wait -m foundations-maas:openstack -t 14400 --workload # up to four hours! [2018-03-17-01:51:03/deploy]: ERROR:root:filebeat/1 failed: workload status is error

juju-crashdump-08d091c8-c98c-4771-84c2-b81dc2f379f4.tar.gz

2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed Traceback (most recent call last): 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/var/lib/juju/agents/unit-filebeat-1/charm/hooks/elasticsearch-relation-changed", line 19, in 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed main() 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/var/lib/juju/agents/unit-filebeat-1/.venv/lib/python3.5/site-packages/charms/reactive/init.py", line 72, in main 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed bus.dispatch(restricted=restricted_mode) 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/var/lib/juju/agents/unit-filebeat-1/.venv/lib/python3.5/site-packages/charms/reactive/bus.py", line 375, in dispatch 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed _invoke(other_handlers) 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/var/lib/juju/agents/unit-filebeat-1/.venv/lib/python3.5/site-packages/charms/reactive/bus.py", line 351, in _invoke 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed handler.invoke() 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/var/lib/juju/agents/unit-filebeat-1/.venv/lib/python3.5/site-packages/charms/reactive/bus.py", line 173, in invoke 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed self._action(*args) 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/var/lib/juju/agents/unit-filebeat-1/charm/reactive/filebeat.py", line 79, in push_filebeat_index 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed push_beat_index(host_string, 'filebeat') 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "lib/elasticbeats.py", line 100, in push_beat_index 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed check_call(cmd) 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed File "/usr/lib/python3.5/subprocess.py", line 581, in check_call 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed raise CalledProcessError(retcode, cmd) 2018-03-17 02:00:08 DEBUG elasticsearch-relation-changed subprocess.CalledProcessError: Command '['curl', '-XPUT', 'http://10.244.41.24:9200/_template/filebeat', '-d@/etc/filebeat/filebeat.template.json']' returned non-zero exit status 7 2018-03-17 02:00:08 ERROR juju.worker.uniter.operation runhook.go:113 hook "elasticsearch-relation-changed" failed: exit status 1

kwmonroe commented 6 years ago

FYI, this has been released in charm rev >= 19:

https://jujucharms.com/filebeat/19