Open akrog79 opened 1 year ago
mmmm, that is very weird. How come you got so many shards??? Are you using datastreams?? Can you give me more details on how your indexes are being generated???
the FortinetDragon works as you say in this repository, but I have ingested a PaloAlto FW and Fortigate FW without datastream, the logstash output generates an index every day.
I recommend you follow the indexing strategy we have in FortiDragon: ILM and datastream per "type". That way you wont make and index per day which is not optimal.
By the way, we also provide a dashboard for palo alto logs 💪
So what we should do? Remove the date on the index name in logstash conf.d file and control it with index templates? I don't know how made a data stream.
I read abaout PaloAltgo on github! when will you post it? I'm looking forward to it, fortidragon is amazing and I haven't configured it 100% yet.
The dashboard for Palo alto is already uploaded to the repo
https://github.com/enotspe/fortinet-2-elasticsearch/blob/master/kibana/panw%20ELK%20850.ndjson
For using our index strategy (ILM and datastreams), you have to follow the instructions provided
https://github.com/enotspe/fortinet-2-elasticsearch#on-kibana
and also setup your output pipeline for using datastreams, like the one provided on
but you don't have PaloAlto integration, no?
No. I have used filebeat module. It is not perfect, but at least it parses the most important logs. The problem is that i don't have an infrastructure where to test it with logstash.
Hello,
I received data and is ingested and procesed fine. But when new day starts, my ingestion give this error:
Validation Failed: 1: this action would add [2] shards, but this cluster currently has [999]/[1000] maximum normal shards open
Any idea?