housekeeping task blocks the worker

[nemesifier]

On instances that contain a lot of data the housekeeping task blocks all the other tasks because it takes long to complete.

The following line:

ActivityHistory.destroy_all(["created_at < ?", time])

Takes very long to complete.

We could either add an index on created_at and/or run the housekeeping more often. I would opt for the second option and implement the first one only if needed.

I took the measure by running this edited version of housekeeping method:

  def housekeeping_measured
    def measure_and_log
        log_file = File.open('./log/housekeeping.txt', 'a')
        start = Time.now
        yield
        finish = Time.now
        diff = finish - start
        puts(diff)
        log_file.puts(diff)
    end
    begin
      time = CONFIG['housekeeping_interval'].months.to_i.ago
      measure_and_log do
        ActivityHistory.destroy_all(["created_at < ?", time])
      end
      measure_and_log do
        AssociatedUserCountHistory.destroy_all(["created_at < ?", time])
      end
      measure_and_log do
        # delete old alerts
        Alert.destroy_all(["created_at < ?", time])
      end
      measure_and_log do
        # build missing property sets
        AccessPoint.build_all_properties()
      end
      measure_and_log do
        # delete orphan property sets
        PropertySet.destroy_orphans()
      end
    rescue Exception => e
      puts "Problem in housekeeping"
      puts "[#{Time.now}] #{e.message}"
      puts "[#{Time.now}] #{e.backtrace.inspect}"
      ExceptionNotifier::Notifier.background_exception_notification(e).deliver
    end
  end

[nemesifier]

Solved.