search

StamusNetworks / SELKS

A Suricata based IDS/IPS/NSM distro
https://www.stamus-networks.com/open-source/#selks
GNU General Public License v3.0
1.44k stars 284 forks source link

🐞🐋 Bug on elasticsearch index mapping for IP fields search #477

Open ulysse31 opened 1 month ago

ulysse31 commented 1 month ago

Is there an existing issue for this?

Current Behavior

Hello,

Via Kibana dashboards (for example : SN-FILE-Transactions), when you try to filter content depending on an IP field, it works without issue with a specific IP (like "dest_ip: 8.8.8.8" or "not dest_ip: 8.8.8.8"), BUT it does not work with CIDR filtering (example "dest_ip: 192.168.0.0/16" or "not dest_ip: 192.168.0.0/16") Which, seems really strange since by default, on all other elasticsearch index mappings, a field IP can always be filtered with CIDR (eg. elasticsearch documentation https://www.elastic.co/guide/en/elasticsearch/reference/7.17/ip.html) On the SELKS docker implementation at least, if we try to filter with "dest_ip: 192.168.0.0/16" we get shard exeption errors as follow :

query_shard_exception at shard 0index logstash-fileinfo-2024.07.30node zG64wDeYSXqMvXshrIkLiA

Type query_shard_exception Reason Failed to parse query [dest_ip: 192.168.0.0/16] Index uuid S1qgMZ3GTP63zrkDFacSqA Index logstash-fileinfo-2024.07.30 Caused by type parse_exception Caused by reason Cannot parse 'dest_ip: 192.168.0.0/16': Lexical error at line 1, column 24. Encountered: after : "/16" Caused by caused by type token_mgr_error Caused by caused by reason Lexical error at line 1, column 24. Encountered: after : "/16"

This type of error induce a potencial error on index mapping creation, or index template usage, or index mapping ... inside the elasticsearch database settings.

Expected Behavior

being able to filter IP fields using CIDR filters.

Steps To Reproduce

  1. Install SELKS docker implementation
  2. ingest data from network (generate network data to feed suricata)
  3. search data on elasticsearch trying to filter IP fields with CIDR mask

Docker version

Docker version 27.1.1, build 6312585

Docker version

Docker Compose version v2.29.1

OS Version

Debian GNU/Linux 12 (bookworm)

Content of the environnement File

COMPOSE_PROJECT_NAME=selks INTERFACES= -i bond1 ELASTIC_MEMORY=64G SCIRIUS_SECRET_KEY= PWD=${PWD}

Version of SELKS

commit 4af455cd15f69f2ba471fa6cd0b96d6aae6e93b9 (HEAD -> master, origin/master, origin/HEAD) Author: Peter Manev pmanev@stamus-networks.com Date: Thu Jun 13 13:18:18 2024 +0200

docker: Add Logstash/Kibana docker versions

Anything else?

No response

pevma commented 1 month ago

You have the correct reason , most likely it is a mapping issue, will need to be investigated.