search

splunk / rwi_executive_dashboard

Splunk Remote Work Insights - Executive Dashboard
https://splunkbase.splunk.com/app/4952
Apache License 2.0
40 stars 13 forks source link

timestamp recognition #29

Open wryanthomas opened 4 years ago

wryanthomas commented 4 years ago

I'm referencing a post I just made to Splunk's Community site: https://community.splunk.com/t5/Getting-Data-In/Zoom-logs-and-Timestamps/m-p/514180#M87145

Upshot: Is there any plan to do more targeted timestamp recognition? I.e., so that the timestamp fields in the events themselves are used for the value assigned to _time?

Apologies if this is not the right place to post this. I'm not sure where to post, whom to reach out to.

l00py commented 4 years ago

Hi @wryanthomas,

For Zoom Webhook, some of the events cannot rely on timestamp fields within the raw data itself, hence the reasons to use DATETIME_CONFIG = NONE. This will get the timestamp closest to the input - input timestamp.

A good example of webhook event: meeting.created, you cannot use the start_time field as the event timestamp should be more about: when was that meeting created.

Hope that answers your question.

Thanks,

Philippe

wryanthomas commented 4 years ago

Hi Philippe.

It does answer the question -- and I do see (it's evident in the report from the SPL I provided) that a number of 'types' of events (such as meeting.create) don't have timestamps.

But... we're having trouble keeping the webhook input stream up ... which leads to bad timestamps. Obviously, we're working to remedy that, but still...

So... it seems like we need to get Zoom to put better (and, ideally, more consistent) timestamp values in their events. Or, alternatively, use sourcetypes to break up by 'type' so we can use good timestamp values that have them, and DATETIME_CONFIG = NONE for those events that don't have it.

Do you know if anyone from Splunk is working on this with Zoom? I thought I heard you mention in an earlier thread (if not on our earlier call a few weeks back) about looking at different sourcetypes. Is that still on the roadmap?

Ryan

From: Philippe Tang notifications@github.com Sent: Thursday, August 20, 2020 2:57 PM To: splunk/rwi_executive_dashboard rwi_executive_dashboard@noreply.github.com Cc: wryanthomas wryanthomas@gmail.com; Mention mention@noreply.github.com Subject: Re: [splunk/rwi_executive_dashboard] timestamp recognition (#29)

Hi @wryanthomashttps://github.com/wryanthomas,

For Zoom Webhook, some of the events cannot rely on timestamp fields within the raw data itself, hence the reasons to use DATETIME_CONFIG = NONE. This will get the timestamp closest to the input - input timestamp.

A good example of webhook event: meeting.created, you cannot use the start_time field as the event timestamp should be more about: when was that meeting created.

Hope that answers your question.

Thanks,

Philippe

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/splunk/rwi_executive_dashboard/issues/29#issuecomment-677872356, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AISEYL73ZKOJHO26G7K7CQ3SBV53BANCNFSM4P7YMAPQ.