flipp-oss / deimos

Framework to work with Kafka, Avro and ActiveRecord
Other
59 stars 22 forks source link

KafkaSource crashes on bulk-import if import hooks are disabled #73

Closed colinmroberts closed 3 years ago

colinmroberts commented 4 years ago

If you have a KafkaSource with import hooks disabled:

  def self.kafka_config
    {
      update: true,
      delete: true,
      import: false,
      create: true
    }
  end

Doing a bulk import that only updates (no inserts) on that class will fail with the following error:

NoMethodError: undefined method `num_inserts' for nil:NilClass

This seems to be because the early return in the import_without_validations_or_callbacks monkey patch does not return the results from super

        results = super
        return unless self.kafka_config[:import] #### Here
        return if array_of_attributes.empty? #### And here

The activerecord-import gem later tries to use the results of the monkey patched method like this and crashes (because it's nil). See import.rb:735

      return_obj.num_inserts = 0 if return_obj.num_inserts.nil?