securityonion-elastic: RC4

[dougburks]

• [x] Elasticsearch

  • [x] Elasticsearch 6.2.3
  • [x] move to jvm.options for setting heap size
  • [x] set dynamic mapping to false - commit

• [x] Logstash

  • [x] Logstash 6.2.3
  • [x] don't automatically install redis files so we can avoid overwriting on package upgrades
  • [x] allow users to disable individual files in /etc/logstash/conf.d/ without them being re-enabled
  • [x] remove NIDS rule lookup to avoid performance hit
  • [x] some outputs have template references like ./logstash-template.json (remove the dot)
  • [x] heap size should be 25% of RAM up to 4GB
  • [x] comment out pipeline.workers: 1 so that logstash can set workers automatically
  • [x] use default logstash.yml and set path.config, http.host, and path.logs
  • [x] move to jvm.options for setting heap size
  • [x] fix kerberos client_cert_subject
  • [x] fix RADIUS parser
  • [x] fix OSSEC sysmon parser
  • [x] fix pfsense parser

• [x] Kibana

  • [x] Kibana 6.2.3
  • [x] so-elastic-configure-kibana-config should lower case hostname when setting seeds
  • [x] Sysmon - Event Type Visualization should be changed to Event ID
  • [x] ingest as many data types as possible (pfsense, beats, ossec, etc.) before exporting dashboards

• [x] CapMe

  • [x] display log that user is pivoting from
  • [x] if pivoting from NIDS alert, display NIDS rule that generated alert
  • [x] check to see if uid is an array

• [x] ElastAlert

  • [x] add flatline rule
  • [x] add new_term rule
  • [x] add change rule

• [x] elasticdownload.conf

  • [x] remove git settings since they're no longer used - commit

• [x] so-*

  • [x] finish so-COMPONENT-VERB control scripts (example: so-logstash-restart)

• [x] so-elastic-status

  • [x] check to see if Logstash is still initializing

• [x] so-import-pcap

  • [x] note that it may take 30 seconds for logs to appear in Kibana

• [x] sosetup-elastic

  • [x] configure remote elastic nodes with skip_unavailable: true

• [x] so-crossclustercheck

  • [x] reconfigure to just output _cluster/settings since we're now using skip_unavailable: true

[dougburks]

submitted for testing: https://groups.google.com/d/topic/security-onion-testing/WU5Sj88nF6s/discussion

[dougburks]

Published: https://blog.securityonion.net/2018/03/elastic-623-and-securityonion-elastic.html