search

counteractive / o365beat

Elastic Beat for fetching and shipping Office 365 audit events
Other
66 stars 27 forks source link

Error Parsing @timestamp #19

Closed Z3r01mPact closed 4 years ago

Z3r01mPact commented 4 years ago

Hiya,

Noticed since i did a update to 7.4.2 i'm now getting these messages:

Nov 28 08:49:39 elkserver o365beat[94399]: 2019-11-28T08:49:39.177Z ERROR instance/beat.go:877 Exiting: parsing time "" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "2006"

Tried adjusting the : # - {from: "CreationTime", to: "", type: ""} # @timestamp to - {from: "CreationTime", to: "", type: string} # @timestamp in order to try and get something through but it still fails with same error. Have you seen this before ?

chris-counteractive commented 4 years ago

Thanks for opening an issue, I've not seen this before. The creation of the @timestamp field actually happens within the beat itself, not via the config file (I should remove that commented line, it's a bit confusing and was designed for possible future use).

To help with the debugging, could you answer a couple questions for me?:

  1. Was it elasticsearch you updated to 7.4.2?
  2. What version of the beat are you using, and on what platform?

If you'd rather send this information via email, I'm at chris@counteractive.net. Thanks again, I'm sure we'll be able to sort this out.

Z3r01mPact commented 4 years ago

ELK stack updated to 7.4.2 last week O365beat version is latest .deb downloaded from your github

Running on Ubuntu 18.04 Vm system... pretty much everything else works, all other collectors are working, just this error in Beats

chris-counteractive commented 4 years ago

Reviewing this, I think I know the problem: I'd bet your state file exists but contains an empty string instead of a date stamp (or in addition to a date stamp). Currently the beat knows what to do if the state file is missing, but then trusts the contents to be properly formed and sensible after that (see the lack of detailed error checking in that part of the beat). That's a bug. A better approach is probably to delete the current state file and go back the full time (7 days) from the time you run the beat: it risks shipping duplicate events, but that seems better than dropping events. At the very least we should print an informative error message and let the user either delete the state file or fix it in their preferred way (they'd more likely know how it was messed up) ... this is the more conservative approach.

An interim fix is to delete the state file (whatever is listed in the registry_file_path key, by default ./o365beat.state), which will force the beat to make a new one with correct content.

Would you be able to confirm this is true in your setup? Thanks again for the feedback.

chris-counteractive commented 4 years ago

I'm going to push a fix for the bug I mentioned and note that it fixes this issue. The fix handles a corrupted state file the same way as a missing one (namely, go back as early as you can to get events flowing, and the state-writing logic will overwrite the broken state file). This programmatically implements the same fix I suggested to you: just delete the current state file. I'll update this issue when I push a new release, but since you can manually implement the fix there should be no need to rush a release just for this issue.

If you delete the state file and still get the error, please let me know and I'll re-open this and investigate further. Thanks again!