search

Azure / logstash-output-kusto

Logstash output for Kusto
Apache License 2.0
13 stars 14 forks source link

Don't send files that are zero bytes #38

Closed KhaoticMind closed 2 years ago

KhaoticMind commented 2 years ago

We have configured our output to a pattern that goes to the second and sometimes there is no message on a second, but the kusto output send the file and asks for ADX do ingest it anyway. This generates some superfluous ingestion failures with the message "BadRequest_NoRecordsOrWrongFormat: The input stream produced 0 bytes. This usually means that the input JSON stream was ill formed." .

So, if the output could check the size of file before sending it to the server would be great.

AsafMah commented 2 years ago

We'll look into implementing it specifically for kusto, but for now you can use pure logstash filters - https://discuss.elastic.co/t/logstash-if-message-is-empty-drop-the-entire-row/221986/2 if ($raw_event =~ /^\s*$/) {drop ();}

yihezkel commented 2 years ago

Hi @KhaoticMind, We're still looking into the fix, but did the workaround resolve your issue?

ag-ramachandran commented 2 years ago

Version 1.0.4 addresses this issue. 0 size files are not transmitted