logstash-plugins / logstash-integration-rabbitmq

Logstash Integration Plugin for RabbitMQ, including Logstash Input and Output Plugins
Apache License 2.0
5 stars 25 forks source link

Rabbitmq output plugin : dynamic values on headers not working as type is Hash not String #42

Open DIFRIN opened 3 years ago

DIFRIN commented 3 years ago

https://github.com/logstash-plugins/logstash-integration-rabbitmq/blob/1d8117ccf3524aa80079d4ad376bf47e8c671389/lib/logstash/outputs/rabbitmq.rb#L146 With this headers attributs of message_properties will fall all time in the constant_properties because it's not of type string

A Solution could be to extract constant_properties[:headers] then do the same as the variable_properties and reseting it in constant_properties before freezing. Something like

` def buildHeaders(event)

      if @constant_properties[:headers]

        constant_headers = @constant_properties[:headers].reject { |_, v| templated?(v) }
        variable_headers = @constant_properties[:headers].select { |_, v| templated?(v) }

        headers = variable_headers.each_with_object(constant_headers.dup) do |(k, v), memo|
          memo.store(k, event.sprintf(v))
        end

        @constant_properties[:headers] = headers
        @constant_properties.freez
      end
    end`

@yaauie could you help us fix this issue ? Thank you in advance

gkoudjou commented 3 years ago

Hi,

I've open roughly the same issue some months ago --> https://github.com/logstash-plugins/logstash-integration-rabbitmq/issues/27

Thanks for the fix and great job for the plugin

bahrar commented 3 years ago

Hello I encountered exactly the same problem on 7.1.0 and 7.1.1 versions : headers map values ​​(properties -> headers -> field value) are not templated (only properties fields of type string are templated)

example to reproduce: input: id-var="id-value"

output{
      rabbitmq {
          exchange => "my-exchange"
          exchange_type => "topic"
          user => "my-user"
          password => "my-password"
          key => "%{[@metadata][routing_key]}"
          host => "my-host"
          port => 35672
          properties => {
            headers => {
               id => "%{[id-var]}"
               version => "1.0"
             }
          }
      }
}

expected message body in rabbitmq : {"id" : "id-value", "date" : "2021-02-03-15.31.32.708295"} but was : {"id" : "%{[id-var]}", "date" : "2021-02-03-15.31.32.708295"}

Thank you in advance for the fix

DIFRIN commented 3 years ago

@jsvd @yaauie coud you please have a look into this issue.