search

counteractive / o365beat

Elastic Beat for fetching and shipping Office 365 audit events
Other
66 stars 27 forks source link

[ModifiedProperties] Can't get text on a START_OBJECT #18

Closed borg333 closed 4 years ago

borg333 commented 4 years ago

Have this in log: Nov 26 11:29:12 logstash logstash[46377]: [2019-11-26T11:29:12,410][WARN ][logstash.outputs.elasticsearch][main] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"o365-2019.11.26", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x52d5bf53>], :response=>{"index"=>{"_index"=>"o365-2019.11.26", "_type"=>"_doc", "_id"=>"-QrUpm4BlECIBFDuKB-8", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [ModifiedProperties] of type [text] in document with id '-QrUpm4BlECIBFDuKB-8'. Preview of field's value: '{NewValue=SharingLinks.7bv3e4a1-cfd3-77nb-5ac4-5fef5e7cbb87.OrganizationEdit.e64cw741-123d-5dff-7375-12e76345f0cy, Name=Name}'", "caused_by"=>{"type"=>"illegal_state_exception", "reason"=>"Can't get text on a START_OBJECT at 1:1421"}}}}} dont know, is is normal or not

chris-counteractive commented 4 years ago

Thanks for the issue, this is not normal, no. Could you confirm which version you're using? We thought the latest update in v1.4.2 fixed this situation, but if you're using the latest and still getting this error I'll need to look into it a bit more. Sorry for the inconvenience, I look forward to helping you sort this out.

chris-counteractive commented 4 years ago

@borg333 - were you able to solve solve this by updating to the latest version? We want to make sure everything's working well for you.

Please see also my response in issue #23, you may have a duplicate processors section in your config that's shadowing the line that fixes this problem. Commenting it out, deleting it, or merging it, will be sure all the processors are working for you.

Thanks!

chris-counteractive commented 4 years ago

This should be long-since sorted using the convert processor, but also there's now support for the script processor in release v1.5.1, along with docs in the README. This would allow you to arbitrarily work with the values in the ModifiedProperties array on the client side, similar to the name-value processing discussed in #41. Closing, pending any additional follow-up or diagnostic info. Thanks!