I'm encountering an issue while attempting to ingest logs from a Fortigate Firewall stored in an S3 bucket into OpenSearch. Here's a breakdown of my setup and the steps I've taken:
Setup: I have two S3 bucket configured for my SIEM, which originally contained logs for AWS services like ELB, guard duty, CloudTrail, security hub etc). Recently, I added an additional bucket/prefix within the same S3 for FortiGate logs.
File Format: The logs in the S3 bucket are in .gz format.
Configuration Changes: I updated the aws.ini file with the following information:
destination.ip = dstip
destination.port = dstport
source.ip = srcip
source.port = srcport
Expected Outcome: After deploying these changes, I expected OpenSearch to create an index and ingest the logs automatically.
However, despite these configurations, I haven't seen any index created in OpenSearch, and there's no data being ingested.
I've tried creating my own index and index pattern, but still, no data is being ingested into OpenSearch.
Could someone please guide me on the necessary steps to successfully ingest additional logs or third-party logs from S3 into OpenSearch? Any insights or suggestions would be greatly appreciated.
Hello GitHub Community,
I'm encountering an issue while attempting to ingest logs from a Fortigate Firewall stored in an S3 bucket into OpenSearch. Here's a breakdown of my setup and the steps I've taken:
Setup: I have two S3 bucket configured for my SIEM, which originally contained logs for AWS services like ELB, guard duty, CloudTrail, security hub etc). Recently, I added an additional bucket/prefix within the same S3 for FortiGate logs. File Format: The logs in the S3 bucket are in .gz format. Configuration Changes: I updated the aws.ini file with the following information:
[Fortigatefirewall] index_name = log-aws-Fortigatefirewall s3_key = firewall/ file_format = json timestamp_key = datetime
'2021-03-08T09:24:27.005808+0000'
ecs = destination.ip destination.port source.ip source.port
destination.ip = dstip destination.port = dstport source.ip = srcip source.port = srcport Expected Outcome: After deploying these changes, I expected OpenSearch to create an index and ingest the logs automatically. However, despite these configurations, I haven't seen any index created in OpenSearch, and there's no data being ingested.
I've tried creating my own index and index pattern, but still, no data is being ingested into OpenSearch.
Could someone please guide me on the necessary steps to successfully ingest additional logs or third-party logs from S3 into OpenSearch? Any insights or suggestions would be greatly appreciated.
Thank you in advance for your help!