search

fastly / sigsci-splunk-app

Splunk app for Fastly (Signal Sciences)
https://splunkbase.splunk.com/app/3495
MIT License
9 stars 10 forks source link

Delay on log ingestion #16

Closed baty0man closed 4 years ago

baty0man commented 4 years ago

Hi,

I'm having an issue with a delay of 5 minutes when ingesting logs from SigSci WAF to our Splunk instance. Basically, any search that I do on the "sigsci" index doesn't return any results for the last 5 minutes.

Capture2

From what I understand, by design, the application retrieve the WAF logs every 5 minutes and send them to Splunk. I tried to change the "delta" and "interval" in the Data Input settings for some of the sourcetypes to 2 minutes, but it still not showing the logs of the last 5 minutes.

I was wondering if maybe I'm missing something in the settings on getting the logs to show up in splunk in less than 5 minutes?

Thanks in advance for your help

dacoburn commented 4 years ago

@yo-blbn that isn't actually an issue with the SPlunk app but actually a restriction of the Signal Sciences API.

https://docs.signalsciences.net/developer/extract-your-data/#timespan-restrictions

The API only allows you to grab the requests from 5 minutes ago until 24 hours ago. There currently isn't any way around this externally.