duplicate_email needs optimization

[eladg]

When generating the queue list, we check for duplicate emails using the following code:

def collect_orders
  data = []
  Resque.redis.lrange("queue:tickets_queue", 0, -1).each do |args|
  ...
    data << [ "TicketsQueue", ip, timestamp, email, ip_appearance ] unless duplicate_email? data, email
  end
end

and:

def duplicate_email?(data, email)    
  data.each { |e| return true if e[3] == email }
  false
end

Technically, an O(N^2) solution. This is only acceptable since generate the list happens once or twice for each sale and can be done completely separately on the background. Would be nice having a better, faster solution.

[adamdelman]

If we don't mind bringing in additional external dependencies, we could simply use a trie (e.g. https://github.com/gonzedge/rambling-trie) to store the email addresses for the purpose of duplicate checking only.

[eladg]

Thanks @adamdelman, I would stick to what we have or simple ruby/resque based solution and not an additional data structure solution.

Revisiting this, I've found those 2 Resque based solutions: resque/resque-loner plugin - example Simple method like: https://gist.github.com/RLovelett/8735865