search

fluent-plugins-nursery / fluent-plugin-concat

Fluentd Filter plugin to concatenate multiline log separated in multiple events.
MIT License
108 stars 33 forks source link

Concatenate various fields #113

Open MarcK2 opened 2 years ago

MarcK2 commented 2 years ago

Problem

Is it possible to concatenate more than one field ? For the key value ? And is it possible to choose the order of concatenation. For example I have line1 with result field INPROGRESS and line2 with esult field SUCCESS. I would like to put line1 StartUserName in line 2 . With my actual configuration It took line1 as a reference and add line2 field in. ...

here is my config

<source>
  @type http
  @id input_http
  port 8888
</source>

<filter jks-build.**>
  @type concat
  key startedUserName
  separator "|"
  n_lines 2
  use_first_timestamp true
</filter>

<match jks-build.**>
 @type stdout
</match>

Expected Behavior

Line 1
{"ciUrl":"url","jobName":"Test-job","fullJobName":"Test-job","number":18,"slaveInfo":{"slaveName":"master","executor":"1","label":"master"},"startTime":1645523967664,"endTime":1645523967665,"startedUserId":"name","startedUserName":"name","result":"INPROGRESS","duration":0,"parameters":{},"scmInfo":{"url":"","branch":"","commit":""},"queueTime":2,"buildUrl":"job/Test-job/18/","contextId":258453088,"buildCause":"Started by user name","buildFailureCauses":[]}

Line 2 
{"ciUrl":"url","jobName":"Test-job","fullJobName":"Test-job","number":18,"slaveInfo":{"slaveName":"","executor":"","label":""},"startTime":1645523967696,"endTime":1645523967696,"startedUserId":"","startedUserName":"","result":"SUCCESS","duration":29,"parameters":{},"scmInfo":{"url":"","branch":"","commit":""},"queueTime":0,"buildUrl":"job/Test-job/18/","contextId":0,"buildCause":"","buildFailureCauses":[]}

Output 
{"ciUrl":"url":"Test-job","fullJobName":"Test-job","number":18,"slaveInfo":{"slaveName":"master","executor":"0","label":"master"},"startTime":1645701396629,"endTime":1645701396633,"startedUserId":"name","startedUserName":"name|","result":"INPROGRESS","duration":0,"parameters":{},"scmInfo":{"url":"","branch":"","commit":""},"queueTime":1,"buildUrl":"job/Test-job/41/","contextId":82069267,"buildCause":"Started by user name","buildFailureCauses":[]}

The ouput I want 
{"ciUrl":"url","jobName":"Test-job","fullJobName":"Test-job","number":18,"slaveInfo":{"slaveName":"","executor":"","label":""},"startTime":1645523967696,"endTime":1645523967696,"startedUserId":"","startedUserName"name"","result":"SUCCESS","duration":29,"parameters":{},"scmInfo":{"url":"","branch":"","commit":""},"queueTime":0,"buildUrl":"job/Test-job/18/","contextId":0,"buildCause":"","buildFailureCauses":[]}

...

Your environment