search

Security-Onion-Solutions / securityonion

Security Onion is a free and open platform for threat hunting, enterprise security monitoring, and log management. It includes our own interfaces for alerting, dashboards, hunting, PCAP, detections, and case management. It also includes other tools such as osquery, CyberChef, Elasticsearch, Logstash, Kibana, Suricata, and Zeek.
https://securityonion.net
3.15k stars 489 forks source link

[esaggs] Object no longer exists in the index pattern #5065

Closed Nicalicious closed 2 years ago

Nicalicious commented 3 years ago

Error in Kibana on fresh install. This issue was posted on the discussion side but seems to be more of an issue rather than a discussion topic. Discussion #4714

Error on several of the dashboards within Kibana. RDP/SMB/Intel/IRC/Modbus/MySQL/NTLM/PE/RADIUS/RDP/and so on...

-Fresh install from latest .iso 2.3.61 -ran sudo soup -replaced certificate with self-signed certificate /etc/pki/managerssl.crt and /etc/pki/managerssl.key

Everything else is default installation for Standalone.

-vcenter/esxi build -2 NICs /one for management / one for promiscuous mode -32GB RAM -24 cpu xeon X5670 -SSD storage kibana error

Nicalicious commented 3 years ago

index patterns After trying to find the index patterns that are missing, this also arises. "No matching indices found: No indices match pattern ":logstash-"

Nicalicious commented 3 years ago

I tried manually adding the keyword to the indices radius add . It removes the error but does not correct the issue and shows no data for the dashboard/card. I added the keyword for RADIUS as it is easier to replicate.

datlife commented 3 years ago

I ran into similar issue as well.

AFIK, the team is currently migrating logstash to ElasticSearch ingest. Therefore, there is no data in *:logstash* as you see in the UI.

One hack is to modify data source in the UI Dashboard to use *:so-* index instead.

Ref: https://docs.securityonion.net/en/2.3/logstash.html#logstash-parsing

Since Logstash no longer parses logs in Security Onion 2, modifying existing parsers or adding new parsers should be done via Elasticsearch.

Nicalicious commented 3 years ago

:so- is the default on the fresh install. What I find odd is there only seems to be a few people with this issue. You would think this would be a much wider-known issue.

RonV666 commented 2 years ago

:so- is the default on the fresh install. What I find odd is there only seems to be a few people with this issue. You would think this would be a much wider-known issue.

I just installed a standalone version from the 2.3.70 .iso and I am having the same issue; most of the dashboards are looking to use the logstash index, which evidently is gone. So most of the dashboards (most of which seem to have the 16.04 version in the dashboard name???) are not able to visualize any data.

How did you fix?

The 'hack' above where it is suggested to change the UI dashboard data source, i've been poking around and have not been able to find how to correct this. I have a bunch of dashboards that show up as error, and as you say, this surprising as this is the default install that few are reporting it.

RonV666 commented 2 years ago

I just tried a non-iso install (CentOS7 minimal + git clone and install, choosing the standalone option). I get the same results, a bunch of dashboards throw an error and complain about the object no longer existing in the index pattern. @Nicalicious did you ever get your dashboards working?

dougburks commented 2 years ago

I think there are really 2 different issues going on here:

  1. Security Onion 2 includes 16.04 dashboards for folks that perform an in-place upgrade from Security Onion 16.04 to Security Onion 2. These dashboards are only designed for older 16.04 data left over from before the upgrade. Any data recorded after the upgrade would be visible on the new Security Onion 2 dashboards. Documented here: https://docs.securityonion.net/en/2.3/kibana.html#dashboards

  2. On the new Security Onion 2 dashboards, some folks are seeing Error on some visualizations where there is no data found. This is due to a recent change in how Kibana works. In previous versions, the visualization would just be blank. In recent versions, it shows Error but it is really nothing to be concerned about. Read more here: https://github.com/Security-Onion-Solutions/securityonion/discussions/4714

Closing this issue as I don't think there is anything to change at this point.

If you have further questions or problems, please start a new discussion at https://securityonion.net/discuss.