Scirius can't get data from ES 6.2.3

[v0yager]

Hello,guys. I am new to this, so I ask for an apology if the question is too easy.

Scirius can't get data from ES 6.2.3 ! But the status of the ES is green light, I can see the indexes in the scirius UI.

I think that scirius query in ES failed ,maybe~ what name I should set in the (scirius -> suricata -> edit), I am confused。

Anyone can help me ,thanks a lot!

[pevma]

Try adjusting the following :

ELASTICSEARCH_VERSION = 6
ELASTICSEARCH_KEYWORD = "keyword"
KIBANA_VERSION = 6
KIBANA_INDEX = ".kibana"
KIBANA_URL = "http://localhost:5601"

in /etc/scirius/local_settings

[v0yager]

I tried ，but failed。 Scirius can jump the page of Kibana dashboard。 But there is no ES data in Scirius UI ElasticSearch like this： USE_ELASTICSEARCH = True ELASTICSEARCH_ADDRESS = "10.3.220.113:9200" ELASTICSEARCH_LOGSTASH_INDEX = "logstash-*" ELASTICSEARCH_LOGSTASH_ALERT_INDEX = ELASTICSEARCH_LOGSTASH_INDEX ELASTICSEARCH_LOGSTASH_TIMESTAMPING = "daily" ELASTICSEARCH_VERSION = 6 ELASTICSEARCH_KEYWORD = "keyword" ELASTICSEARCH_HOSTNAME = "COMDEV0585"

Kibana like this： USE_KIBANA = True KIBANA_PROXY = True KIBANA_URL = "http://10.3.220.113:5601" KIBANA_INDEX = ".kibana" KIBANA_VERSION= 6 KIBANA_DASHBOARDS_COUNT = 20 KIBANA_DASHBOARDS_PATH = '/opt/kibana-dashboards/'

[regit]

Can yoy try to set the remote elasticsearch address in the interface (in Stamus Icon-> settings).

[v0yager]

I have set the remote ES address .But ,there is no data in Scirius UI The eve.json log like this in ES Thanks a lot~

[pevma]

unselect "use elasticsearch" - so you only leave the "custom" option? Any luck? (plus clear browser cache just in case)

[v0yager]

I tried whether selecting "use custom ES" option, Scirius can get index and cluster info from ES,and the status of ES is yellow or green. But there is no ES data in Scirius UI So maybe other problems~

I also set the time-zone as localtime and change the name of the Suricata (scirius -> suricata -> edit) as the hostname in ES events. Maybe the problem is the hostname? Thanks a log agian~~

[pevma]

Can you share your full local_settings ? (privately if you want to) those are the only changes we make to have it working for ELK6 - https://github.com/StamusNetworks/SELKS/blob/SELKS5-WIP/staging/config/hooks/live/chroot-inside-Debian-Live.hook.chroot#L115

[RonnieNiu]

@v0yager hi:
I faced the problem samed with you, haved you solved it ? thanks

[pevma]

Think you would need to adjust the address of the elasticsearch in local_settings as well, did you do that ?

[RonnieNiu]

Yes ,i do. now all status is green ,but no data。

[pevma]

Whats is your local_settings.py look like ?

[ipworkx]

Same problem here. What is the solution?

[pevma]

Whats is your local_settings.py look like ? Inside you should probably set the hostname/es variables correct if not already done. Something similar to - https://github.com/StamusNetworks/SELKS/issues/201#issuecomment-538728806 depending if ES is local or remote.

[ipworkx]

Hi, Sort of the same as above. The only difference is that my index name is different. It is "suricata-" and not "logstash-" It looks like there is some code within that's not using the proposed variable: ELASTICSEARCH_LOGSTASH_INDEX = "suricata-*"

I will update more tonight.

[pevma]

Ok - let us know hoe it goes!

[ipworkx]

My local_settings.py is like:

USE_ELASTICSEARCH = True ELASTICSEARCH_KEYWORD = "keyword" ELASTICSEARCH_LOGSTASH_INDEX = "suricata-*" ELASTICSEARCH_LOGSTASH_ALERT_INDEX = "suricata-alert-" ELASTICSEARCH_LOGSTASH_TIMESTAMPING = "daily" ELASTICSEARCH_VERSION = 6 ELASTICSEARCH_KEYWORD = "keyword" ELASTICSEARCH_HOSTNAME = "u002.system.ipworkx.loc" ELASTICSEARCH_ADDRESS = "u002.system.ipworkx.loc:9200"

USE_KIBANA = True KIBANA_PROXY = True KIBANA_DASHBOARDS_PATH = '/opt/kibana-dashboards/' KIBANA_VERSION = 6 KIBANA_INDEX = ".kibana_1" KIBANA_URL = "http://u002.system.ipworkx.loc:5601" KIBANA6_DASHBOARDS_PATH = "/opt/selks/kibana6-dashboards/"

SURICATA_UNIX_SOCKET = "/var/run/suricata/suricata-command.socket"

SURICATA_NAME_IS_HOSTNAME = False

USE_EVEBOX = True EVEBOX_ADDRESS = "u002.system.ipworkx.loc:5636"

USE_SURICATA_STATS = True USE_LOGSTASH_STATS = True

ALLOWED_HOSTS=["*"] STATIC_ROOT="/var/lib/scirius/static/"

DATABASES = { 'default': { 'ENGINE': 'django.db.backends.sqlite3', 'NAME': os.path.join(BASE_DIR, 'db', 'db.sqlite3'), } } DBBACKUP_STORAGE_OPTIONS = {'location': '/var/backups/'}

My logstash output config is like:

output { if [event_type] == "stats" { elasticsearch { hosts => [ "172.16.32.2:9200", "172.16.32.5:9200", "172.16.32.7:9200" ] index => "suricata-stats-%{+YYYY.MM.dd}" template => "/etc/logstash/conf.d/suricata/template/suricata-stats-template.json" template_name => "suricata-stats" template_overwrite => true } } else { if [event_type] { elasticsearch { hosts => [ "172.16.32.2:9200", "172.16.32.5:9200", "172.16.32.7:9200" ] index => "suricata-%{event_type}-%{+YYYY.MM.dd}" template => "/etc/logstash/conf.d/suricata/template/suricata-template.json" template_name => "suricata" template_overwrite => true } } }

[ipworkx]

I'm running a SELKS box using filebeat and outputting to kafka, From kafka to Elastic. Kibana works fine

[pevma]

The ES address is reachable form Scirius ?

[ipworkx]

Yep my Scirius machine is running ip 172.16.32.9 which is in the same network. Scirius is seeing all indices from the cluster. All is green and looks fine. I guess.

What values has to be entered in the suricata screen in scirius? Currently I have: Name: u002.system.ipworkx.loc Description: u002.system.ipworkx.loc Rules Directory: /etc/suricata/rules Suricata Configuration File: /etc/suricata/suricata.yaml . . . Is there a way I can t-shoot anywhere?

[ipworkx]

The hunt screen with alerts seems to show alerts. So that's fine, although the dashboard itself does not show anything. Item signatures is also fine. Hope this might give you a clue?

[ipworkx]

Could it be that scirius is looking for a host field in the alert indice? The original host field is now called fields.host due to filebeat and logstash doesn't want to process the host field because of its special purpose

[pevma]

can you try with ELASTICSEARCH_LOGSTASH_ALERT_INDEX = "suricata-alert-*"
then restart Scisiurs/Kibana