search

StamusNetworks / scirius

Scirius is a web application for Suricata ruleset management and threat hunting.
GNU General Public License v3.0
621 stars 148 forks source link

Scirius Suricata displaying no data #166

Closed lukelee1987 closed 5 years ago

lukelee1987 commented 5 years ago

sciriussetting_1 suricata_nodata Hi all, I have the above settings, but I cant really load the data from Suricata and display. Please advice.

pevma commented 5 years ago

Is your logstash log shipping functioning - up and running ?

-- Regards, Peter Manev

On 19 Dec 2018, at 09:03, Sky Luke notifications@github.com wrote:

Hi all, I have the above settings, but I cant really load the data from Suricata and display. Please advice.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

lukelee1987 commented 5 years ago

I think my logstash's log is combined in Suricata eve.json file, if I am not wrong. What I have in the directory (/var/log/logstash) are logstash.err / logstash.stdout / logstash.log.

Besides, I check on the logstash.log, and I still sees this Permission denied issue when I already changed the access rights. image

pevma commented 5 years ago

It seems you have an explicit something is wring with your configuration msg in the logstash log. If logstash is not working - you would probably have no logs in elasticsearch and naturally Scirius can not display anything. First I think you need to fix your logstash config. I am not sure if you have made any changes - but you can use a reference from the default one in SELKS 4 - if you run ES 5.x ( https://github.com/StamusNetworks/SELKS/tree/master/staging/etc/logstash/conf.d ) and SELKS 5 if you rune ES 6.x ( https://github.com/StamusNetworks/SELKS/tree/SELKS5/staging/etc/logstash/conf.d )

lukelee1987 commented 5 years ago

Hi, currently I am using / running ES 6.4. I have changed my HOST to my host ip. and also changed the type => "flow"

pevma commented 5 years ago

You should probable seek around line 12 (as in the err log message) for some closing statement missing (curly bracket or similar) Try with the default SELKS 5 config as well.

lukelee1987 commented 5 years ago

I have tried to remove the comment line. Now seem some issues has been resolved. Now left these. image

Is it due to the host I input? The host ip.

lukelee1987 commented 5 years ago

Is this correct?? pointing to Elasticsearch server:

lukelee1987 commented 5 years ago

image

pevma commented 5 years ago

@lukelee1987 - did you compare to the default logstash templates as suggested by me multiple times ?

lukelee1987 commented 5 years ago

@pevma Yes. I have also changed the host IP to the original one.

image

pevma commented 5 years ago

Not sure - it seems fine here - https://github.com/StamusNetworks/SELKS/blob/SELKS5/staging/etc/logstash/conf.d/logstash.conf#L80. I would suggest to conftest the original SELKS5 logstash conf with your set up and see if any different. You should also have only one conf file in that folder (not multiples).

lukelee1987 commented 5 years ago

I saw some other discussion on this topic too, explaining that there might be multiple directories for Logstash. Do you know how can I identify them and remove it? Thanks.

pevma commented 5 years ago

What discussion are you referring to ?

lukelee1987 commented 5 years ago

https://stackoverflow.com/questions/39082203/unknown-setting-hosts-for-elasticsearch Looks like they facing the same issue.

pevma commented 5 years ago

Looks like they removed the multiple directories (or configs in the same folder) and that solved - based on whats written there.

lukelee1987 commented 5 years ago

Gosh ..... my apologies, my logstash is version 1. Now seems ok. Is this correct?

image

lukelee1987 commented 5 years ago

since I am using the oldest version 1.5.3. Do you think I should upgrade it to the latest one? Is there a shortcut of doing it?

pevma commented 5 years ago

Yes please - 1.x is not supported neither from Logstahs nor from us. I would suggests to recheck your whole set up as well. Thanks for reporting back.

lukelee1987 commented 5 years ago

I have upgraded the Logstash version. But there is still no data coming in to Suricata.

pevma commented 5 years ago

Is this a SELKS installation ?

lukelee1987 commented 5 years ago

Kind of. But I installed it separately. Scirius, Elasticsearch, Logstash, Kibana and Suricata. Now my Kibana's Visualize not showing Data.

image

pevma commented 5 years ago

Then it is probably best if you follow up the instructions by Elasticsearch docs of how to do that for the kind of OS you are using. (example - https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html)

Until the installations is done and the components configured (SELKS does that done by default) properly there most likely would be no data available in the dashboards / Kiabna and Scirius respectively.

lukelee1987 commented 5 years ago

Is there a way I can test if these platforms are able to communicate to each other, like reading data. I cant ensure there are data and how they are being transferred. Thanks.

pevma commented 5 years ago

yes - follow the documentation , use the same configs as in SELKS and check for any errors in the respective logs. Once that is in place with no errors - you should be seeing data i think.

lukelee1987 commented 5 years ago

Pevma, the challenge is that I installed SELKS separately. And there are too many configurations need to be taken care of.