search

robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.48k stars 595 forks source link

kibana not starting #332

Closed isriam closed 5 years ago

isriam commented 5 years ago

i did a git clone of elastiflow then did a docker-compose up -d. kibana doesn't want to start and has the following errors:

{"type":"log","@timestamp":"2019-05-18T23:08:08Z","tags":["error","elasticsearch","admin"],"pid":1,"message":"Request error, retrying\nHEAD http://127.0.0.1:9200/ => connect ECONNREFUSED 127.0.0.1:9200"},
{"type":"log","@timestamp":"2019-05-18T23:08:08Z","tags":["status","plugin:elasticsearch@undefined","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"},
{"type":"log","@timestamp":"2019-05-18T23:08:46Z","tags":["fatal","root"],"pid":1,"message":"{ [search_phase_execution_exception] all shards failed :: {\"path\":\"/.kibana/_count\",\"query\":{},\"body\":\"{\\\"query\\\":{\\\"bool\\\":{\\\"should\\\":[{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"visualization\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.visualization\\\":\\\"7.0.1\\\"}}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"dashboard\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.dashboard\\\":\\\"7.0.0\\\"}}}}]}},{\\\"bool\\\":{\\\"must\\\":[{\\\"exists\\\":{\\\"field\\\":\\\"search\\\"}},{\\\"bool\\\":{\\\"must_not\\\":{\\\"term\\\":{\\\"migrationVersion.search\\\":\\\"7.0.0\\\"}}}}]}}]}}}\",\"statusCode\":503,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[]},\\\"status\\\":503}\"}\n    at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15)\n    at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7)\n    at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)\n    at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4935:19)\n    at IncomingMessage.emit (events.js:194:15)\n    at endReadableNT (_stream_readable.js:1103:12)\n    at process._tickCallback (internal/process/next_tick.js:63:19)\n  status: 503,\n  displayName: 'ServiceUnavailable',\n  message: '[search_phase_execution_exception] all shards failed',\n  path: '/.kibana/_count',\n  query: {},\n  body:\n   { error:\n      { root_cause: [],\n        type: 'search_phase_execution_exception',\n        reason: 'all shards failed',\n        phase: 'query',\n        grouped: true,\n        failed_shards: [] },\n     status: 503 },\n  statusCode: 503,\n  response:\n   '{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}',\n  toString: [Function],\n  toJSON: [Function] }"},
,
 FATAL  [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/_count","query":{},"body":"{\"query\":{\"bool\":{\"should\":[{\"bool\":{\"must\":[{\"exists\":{\"field\":\"visualization\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.visualization\":\"7.0.1\"}}}}]}},{\"bool\":{\"must\":[{\"exists\":{\"field\":\"dashboard\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.dashboard\":\"7.0.0\"}}}}]}},{\"bool\":{\"must\":[{\"exists\":{\"field\":\"search\"}},{\"bool\":{\"must_not\":{\"term\":{\"migrationVersion.search\":\"7.0.0\"}}}}]}}]}}}","statusCode":503,"response":"{\"error\":{\"root_cause\":[],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[]},\"status\":503}"},
,
robcowart commented 5 years ago

Which OS are you testing on? I just tested again on a fresh Ubuntu 18.04 install using the docker-compose.yml that is currently in master and it works fine.

isriam commented 5 years ago

ubuntu 16.04 with docker. let me get the docker version.

robcowart commented 5 years ago

Can you also past your exact docker-compose.yml here?

isriam commented 5 years ago

i just did a git clone of your repo, and then launched it from that docker-compose.yml.

#------------------------------------------------------------------------------
# Portions of this file are Copyright (C)2019 Robert Cowart
# 
# Portions of this file are subject to the Robert Cowart Public License (the
# "License") and may not be used or distributed except in compliance with the
# License. You may obtain a copy of the License at:
# 
# http://www.koiossian.com/public/robert_cowart_public_license.txt
# 
# Software distributed under the License is distributed on an "AS IS" basis,
# WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
# the specific language governing rights and limitations under the License.
# 
# The Original Source Code was developed by Robert Cowart. Portions created by
# Robert Cowart are Copyright (C)2019 Robert Cowart. All Rights Reserved.
#------------------------------------------------------------------------------

version: '3'

services:
  elastiflow-elasticsearch-oss:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.0.1
    container_name: elastiflow-elasticsearch-oss
    restart: 'no'
    ulimits:
      memlock:
        soft: -1
        hard: -1
    network_mode: host
    volumes:
      - /var/lib/elastiflow_es:/usr/share/elasticsearch/data
    environment:
      # JVM Heap size
      #   - this should be at least 2GB for simple testing, receiving only a few flows per second.
      #   - for production environments upto 31GB is recommended.
      ES_JAVA_OPTS: '-Xms2g -Xmx2g'

      cluster.name: elastiflow

      bootstrap.memory_lock: 'true'

      network.host: 0.0.0.0
      http.port: 9200
      discovery.type: 'single-node'

      indices.query.bool.max_clause_count: 8192
      search.max_buckets: 100000

      action.destructive_requires_name: 'true'

  elastiflow-kibana-oss:
    image: docker.elastic.co/kibana/kibana-oss:7.0.1
    container_name: elastiflow-kibana-oss
    restart: 'no'
    depends_on:
      - elastiflow-elasticsearch-oss
    network_mode: host
    environment:
      SERVER_HOST: 0.0.0.0
      SERVER_PORT: 5601

      ELASTICSEARCH_HOSTS: "http://127.0.0.1:9200"

      KIBANA_DEFAULTAPPID: "dashboard/653cf1e0-2fd2-11e7-99ed-49759aed30f5"

      LOGGING_QUIET: 'true'

  elastiflow-logstash-oss:
    image: robcowart/elastiflow-logstash-oss:3.5.0_7.0.1
    container_name: elastiflow-logstash-oss
    restart: 'no'
    depends_on:
      - elastiflow-elasticsearch-oss
    network_mode: host
    environment:
      # JVM Heap size - this MUST be at least 3GB (4GB preferred)
      LS_JAVA_OPTS: '-Xms4g -Xmx4g'

      # ElastiFlow global configuration
      ELASTIFLOW_DEFAULT_APPID_SRCTYPE: "__UNKNOWN"

      # Name resolution option
      ELASTIFLOW_RESOLVE_IP2HOST: "false"
      ELASTIFLOW_NAMESERVER: "127.0.0.1"

      ELASTIFLOW_NETFLOW_IPV4_PORT: 2055
      ELASTIFLOW_SFLOW_IPV4_PORT: 6343
      ELASTIFLOW_IPFIX_TCP_IPV4_PORT: 4739
robcowart commented 5 years ago

Did you create and set permissions to data directory to which the Elasticsearch container is bind mounted?

https://github.com/robcowart/elastiflow/blob/master/DOCKER.md#prepare-the-data-path

isriam commented 5 years ago

yea, i'm going to keep playing with it though. if you can think of anything just let me know. i'll keep troubleshooting until i isolate it.

isriam commented 5 years ago

i had to set the elastisearch_hosts in this line to my local IP for it to work.

ELASTICSEARCH_HOSTS: "http://192.168.1.26:9200"

possible to use aliases for that instead of 127.0.0.1 or local ip?

isriam commented 5 years ago

it works fine in 18.04 ubuntu. not sure why that loopback doesnt work in 16.

robcowart commented 5 years ago

Good to know! Thanks for testing and figuring it out.

ghost commented 4 years ago

I have the the same problem please

My docker compose the ports extra defined to open it for external networks


`version: '3'

services:
  elastiflow-elasticsearch-oss:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:7.3.0
    container_name: elastiflow-elasticsearch-oss
    restart: 'no'
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - /var/lib/elastiflow_es:/usr/share/elasticsearch/data
    ports:
      - "9200:9200"
    environment:
      # JVM Heap size
      #   - this should be at least 2GB for simple testing, receiving only a few flows per second.
      #   - for production environments upto 31GB is recommended.
      ES_JAVA_OPTS: '-Xms3g -Xmx3g'

      cluster.name: elastiflow

      bootstrap.memory_lock: 'true'

      network.host: "0.0.0.0"
      http.port: 9200
      discovery.type: 'single-node'

      indices.query.bool.max_clause_count: 8192
      search.max_buckets: 100000

      action.destructive_requires_name: 'true'

  elastiflow-kibana-oss:
    image: docker.elastic.co/kibana/kibana-oss:7.3.0
    container_name: elastiflow-kibana-oss
    restart: 'no'
    depends_on:
      - elastiflow-elasticsearch-oss
    ports:
      - "5601:5601"
    environment:
      SERVER_HOST: "0.0.0.0"
      SERVER_PORT: 5601

      ELASTICSEARCH_HOSTS: "http://10.0.30.29:9200"

      KIBANA_DEFAULTAPPID: "dashboard/653cf1e0-2fd2-11e7-99ed-49759aed30f5"

      LOGGING_QUIET: 'true'

  elastiflow-logstash-oss:
    image: robcowart/elastiflow-logstash-oss:3.5.1
    container_name: elastiflow-logstash-oss
    restart: 'no'
    ports:
      - "2055:2055"
      - "6343:6343"
      - "4739:4739"
    depends_on:
      - elastiflow-elasticsearch-oss
    environment:
      # JVM Heap size - this MUST be at least 3GB (4GB preferred)
      LS_JAVA_OPTS: '-Xms5g -Xmx5g'
      ELASTIFLOW_ES_HOST: "172.19.0.2"
      # ElastiFlow global configuration
      ELASTIFLOW_DEFAULT_APPID_SRCTYPE: "__UNKNOWN"

      # Name resolution option
      ELASTIFLOW_RESOLVE_IP2HOST: "false"
      ELASTIFLOW_NAMESERVER: "127.0.0.1"

      ELASTIFLOW_NETFLOW_IPV4_PORT: 2055
      ELASTIFLOW_SFLOW_IPV4_PORT: 6343
      ELASTIFLOW_IPFIX_TCP_IPV:4_PORT: 4739`

LOG FROM COMPOSE

`####docker-compose up
Starting elastiflow-elasticsearch-oss ... done
Starting elastiflow-logstash-oss      ... done
Starting elastiflow-kibana-oss        ... done
Attaching to elastiflow-elasticsearch-oss, elastiflow-kibana-oss, elastiflow-logstash-oss
elastiflow-elasticsearch-oss    | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
elastiflow-kibana-oss           | {"type":"log","@timestamp":"2019-09-22T10:13:16Z","tags":["error","elasticsearch","admin"],"pid":1,"message":"Request error, retrying\nGET http://10.0.30.29:9200/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip => connect EHOSTUNREACH 10.0.30.29:9200"}
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:17,188+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/fedora-root)]], net usable_space [216.8gb], net total_space [224.8gb], types [xfs]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:17,219+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "heap size [2.9gb], compressed ordinary object pointers [true]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:17,253+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "node name [918624de194a], node ID [HY3MmE8RSDWf2nrRLsZb-A], cluster name [elastiflow]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:17,264+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "version[7.3.0], pid[1], build[oss/docker/de777fa/2019-07-24T18:30:11.767338Z], OS[Linux/5.2.15-200.fc30.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:17,265+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "JVM home [/usr/share/elasticsearch/jdk]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:17,266+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-4150948664004315653, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Des.cgroups.hierarchy.override=/, -Xms3g, -Xmx3g, -Dio.netty.allocator.type=pooled, -XX:MaxDirectMemorySize=1610612736, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=oss, -Des.distribution.type=docker, -Des.bundled_jdk=true]"  }
elastiflow-kibana-oss           | {"type":"log","@timestamp":"2019-09-22T10:13:17Z","tags":["status","plugin:elasticsearch@7.3.0","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - No Living connections","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,034+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [aggs-matrix-stats]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,038+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [analysis-common]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,046+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [ingest-common]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,046+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [ingest-geoip]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,047+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [ingest-user-agent]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,050+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [lang-expression]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,051+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [lang-mustache]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,051+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [lang-painless]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,052+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [mapper-extras]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,052+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [parent-join]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,052+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [percolator]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,053+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [rank-eval]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,053+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [reindex]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,054+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [repository-url]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,054+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "loaded module [transport-netty4]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:20,055+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "no plugins loaded"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:31,988+0000", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "using discovery type [single-node] and seed hosts providers [settings]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:33,744+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "initialized"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:33,745+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "starting ..."  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,364+0000", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "publish_address {172.19.0.2:9300}, bound_addresses {[::]:9300}"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,414+0000", "level": "WARN", "component": "o.e.b.BootstrapChecks", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,433+0000", "level": "INFO", "component": "o.e.c.c.Coordinator", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "cluster UUID [rBiZpclMSoqQohx8mp77pg]"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,660+0000", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "elected-as-master ([1] nodes joined)[{918624de194a}{HY3MmE8RSDWf2nrRLsZb-A}{FdtGbk3jREmBombdKbiOJw}{172.19.0.2}{172.19.0.2:9300}{dim} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 12, version: 46, reason: master node changed {previous [], current [{918624de194a}{HY3MmE8RSDWf2nrRLsZb-A}{FdtGbk3jREmBombdKbiOJw}{172.19.0.2}{172.19.0.2:9300}{dim}]}"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,795+0000", "level": "INFO", "component": "o.e.c.s.ClusterApplierService", "cluster.name": "elastiflow", "node.name": "918624de194a",  "message": "master node changed {previous [], current [{918624de194a}{HY3MmE8RSDWf2nrRLsZb-A}{FdtGbk3jREmBombdKbiOJw}{172.19.0.2}{172.19.0.2:9300}{dim}]}, term: 12, version: 46, reason: Publication{term=12, version=46}"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,886+0000", "level": "INFO", "component": "o.e.h.AbstractHttpServerTransport", "cluster.name": "elastiflow", "node.name": "918624de194a", "cluster.uuid": "rBiZpclMSoqQohx8mp77pg", "node.id": "HY3MmE8RSDWf2nrRLsZb-A",  "message": "publish_address {172.19.0.2:9200}, bound_addresses {[::]:9200}"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:34,888+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "elastiflow", "node.name": "918624de194a", "cluster.uuid": "rBiZpclMSoqQohx8mp77pg", "node.id": "HY3MmE8RSDWf2nrRLsZb-A",  "message": "started"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:35,023+0000", "level": "INFO", "component": "o.e.g.GatewayService", "cluster.name": "elastiflow", "node.name": "918624de194a", "cluster.uuid": "rBiZpclMSoqQohx8mp77pg", "node.id": "HY3MmE8RSDWf2nrRLsZb-A",  "message": "recovered [1] indices into cluster_state"  }
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:13:36,131+0000", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "elastiflow", "node.name": "918624de194a", "cluster.uuid": "rBiZpclMSoqQohx8mp77pg", "node.id": "HY3MmE8RSDWf2nrRLsZb-A",  "message": "Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.kibana_1][0]] ...])."  }
elastiflow-logstash-oss         | Sending Logstash's logs to /usr/share/logstash/logs which is now configured via log4j2.properties
elastiflow-logstash-oss         | [2019-09-22T10:13:52,082][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
elastiflow-logstash-oss         | [2019-09-22T10:13:52,102][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
elastiflow-logstash-oss         | [2019-09-22T10:13:53,490][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.1.4"}
elastiflow-logstash-oss         | [2019-09-22T10:13:54,193][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
elastiflow-logstash-oss         | [2019-09-22T10:16:53,943][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://elastic:xxxxxx@172.19.0.2:9200/]}}
elastiflow-logstash-oss         | [2019-09-22T10:16:54,515][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://elastic:xxxxxx@172.19.0.2:9200/"}
elastiflow-logstash-oss         | [2019-09-22T10:16:54,877][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>7}
elastiflow-logstash-oss         | [2019-09-22T10:16:54,886][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>7}
elastiflow-logstash-oss         | [2019-09-22T10:16:54,974][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>"/etc/logstash/elastiflow/templates/elastiflow.template.json"}
elastiflow-logstash-oss         | [2019-09-22T10:16:54,986][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//172.19.0.2"]}
elastiflow-logstash-oss         | [2019-09-22T10:16:55,378][INFO ][logstash.outputs.elasticsearch] Index Lifecycle Management is set to 'auto', but will be disabled - Index Lifecycle management is not installed on your Elasticsearch cluster
elastiflow-logstash-oss         | [2019-09-22T10:16:55,386][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"order"=>0, "version"=>30501, "index_patterns"=>"elastiflow-3.5.1-*", "settings"=>{"index"=>{"number_of_shards"=>3,
..............................
keyword"}, "sampling_interval"=>{"type"=>"long"}, "server_addr"=>{"type"=>"ip"}, "server_asn"=>{"type"=>"long"}, "server_autonomous_system"=>{"type"=>"keyword"}, "server_city"=>{"type"=>"keyword"}, "server_country"=>{"type"=>"keyword"}, "server_country_code"=>{"type"=>"keyword"}, "server_geo_location"=>{"type"=>"geo_point"}, "server_hostname"=>{"type"=>"keyword"}, "server_rep_tags"=>{"type"=>"keyword"}, "service_name"=>{"type"=>"keyword"}, "service_port"=>{"type"=>"long"}, "src_addr"=>{"type"=>"ip"}, "src_addr_trans"=>{"type"=>"ip"}, "src_asn"=>{"type"=>"long"}, "src_autonomous_system"=>{"type"=>"keyword"}, "src_city"=>{"type"=>"keyword"}, "src_country"=>{"type"=>"keyword"}, "src_country_code"=>{"type"=>"keyword"}, "src_geo_location"=>{"type"=>"geo_point"}, "src_hostname"=>{"type"=>"keyword"}, "src_mac"=>{"type"=>"keyword"}, "src_mask_len"=>{"type"=>"long"}, "src_port"=>{"type"=>"long"}, "src_port_trans"=>{"type"=>"long"}, "src_port_name"=>{"type"=>"keyword"}, "src_rep_tags"=>{"type"=>"keyword"}, "tcp_flags"=>{"type"=>"keyword"}, "tos"=>{"type"=>"long"}, "traffic_direction"=>{"type"=>"keyword"}, "traffic_locality"=>{"type"=>"keyword"}, "vlan"=>{"type"=>"long"}}}, "node"=>{"dynamic"=>true, "type"=>"object", "properties"=>{"ipaddr"=>{"type"=>"ip"}, "hostname"=>{"type"=>"keyword"}}}, "tags"=>{"type"=>"keyword"}}}}}
elastiflow-logstash-oss         | [2019-09-22T10:36:30,164][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/elastiflow-3.5.1
elastiflow-elasticsearch-oss    | {"type": "server", "timestamp": "2019-09-22T10:36:30,851+0000", "level": "INFO", "component": "o.e.c.m.MetaDataIndexTemplateService", "cluster.name": "elastiflow", "node.name": "bb915980b380", "cluster.uuid": "rBiZpclMSoqQohx8mp77pg", "node.id": "HY3MmE8RSDWf2nrRLsZb-A",  "message": "adding template [elastiflow-3.5.1] for index patterns [elastiflow-3.5.1-*]"  }
elastiflow-logstash-oss         | [2019-09-22T10:36:31,660][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/etc/logstash/elastiflow/geoipdbs/GeoLite2-City.mmdb"}
elastiflow-logstash-oss         | [2019-09-22T10:36:31,703][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/etc/logstash/elastiflow/geoipdbs/GeoLite2-ASN.mmdb"}
elastiflow-logstash-oss         | [2019-09-22T10:36:41,622][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/etc/logstash/elastiflow/geoipdbs/GeoLite2-City.mmdb"}
elastiflow-logstash-oss         | [2019-09-22T10:36:41,624][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/etc/logstash/elastiflow/geoipdbs/GeoLite2-ASN.mmdb"}
elastiflow-logstash-oss         | [2019-09-22T10:36:51,816][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"elastiflow", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500, :thread=>"#<Thread:0x6ea87f71 run>"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,190][INFO ][logstash.pipeline        ] Pipeline started {"pipeline.id"=>"elastiflow"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,373][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:2055"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,387][INFO ][logstash.inputs.tcp      ] Starting tcp input listener {:address=>"0.0.0.0:4739", :ssl_enable=>"false"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,421][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:4739"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,434][INFO ][logstash.inputs.udp      ] Starting UDP listener {:address=>"0.0.0.0:6343"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,541][WARN ][logstash.inputs.udp      ] Unable to set receive_buffer_bytes to desired size. Requested 33554432 but obtained 212992 bytes.
elastiflow-logstash-oss         | [2019-09-22T10:36:52,544][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:2055", :receive_buffer_bytes=>"212992", :queue_size=>"2048"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,573][WARN ][logstash.inputs.udp      ] Unable to set receive_buffer_bytes to desired size. Requested 33554432 but obtained 212992 bytes.
elastiflow-logstash-oss         | [2019-09-22T10:36:52,574][WARN ][logstash.inputs.udp      ] Unable to set receive_buffer_bytes to desired size. Requested 33554432 but obtained 212992 bytes.
elastiflow-logstash-oss         | [2019-09-22T10:36:52,602][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:4739", :receive_buffer_bytes=>"212992", :queue_size=>"2048"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,603][INFO ][logstash.inputs.udp      ] UDP listener started {:address=>"0.0.0.0:6343", :receive_buffer_bytes=>"212992", :queue_size=>"2048"}
elastiflow-logstash-oss         | [2019-09-22T10:36:52,686][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["elastiflow"]}
`
Karlheinzniebuhr commented 4 years ago

Same error on Debian 10

yozshujar commented 4 years ago

Same on CentOS Linux release 7.7.1908