flipp-oss / deimos

Framework to work with Kafka, Avro and ActiveRecord
Other
59 stars 22 forks source link

NoMethodError: undefined method `decoder' #195

Closed 2rba closed 11 months ago

2rba commented 11 months ago

When db_producer catches Kafka::MessageSizeTooLarge exception then log attempt fails with "NoMethodError: undefined method "decoder'" exception while trying to decode message for logging with Deimos::KafkaMessage.decoded(messages)

backtrace:

deimos-ruby-1.22.3/lib/deimos/kafka_message.rb:46:in `block in decoded'",
activerecord-6.1.7.4/lib/active_record/relation/delegation.rb:88:in `each'",
activerecord-6.1.7.4/lib/active_record/relation/delegation.rb:88:in `each'",
deimos-ruby-1.22.3/lib/deimos/kafka_message.rb:43:in `map'",
deimos-ruby-1.22.3/lib/deimos/kafka_message.rb:43:in `decoded'",
deimos-ruby-1.22.3/lib/deimos/utils/db_producer.rb:103:in `rescue in block in process_topic_batch'",

looks like Deimos::KafkaMessage.decoded messed up with .new call a possible solution could be .decoded method updated as:

    def self.decoded(messages=[])
      return [] if messages.empty?

      decoder_class = self.decoder(messages.first.topic)
      decoder = decoder_class&.new
      messages.map do |m|
        {
          key: m.key.present? ? decoder&.decode_key(m.key) || m.key : nil,
          payload: decoder_class&.decoder&.decode(m.message) || m.message
        }
      end
    end
dorner commented 11 months ago

Thanks for the report! PRs are welcome :grin: