soveran / ohm

Object-Hash Mapping for Redis
http://ohm.keyvalue.org
MIT License
1.4k stars 167 forks source link

Race condition on list delete #228

Open chriso0710 opened 6 years ago

chriso0710 commented 6 years ago

Referencing #224 here, so the following applies to the model described there. And I am still using Ohm 2.3 :-|

I might have a race condition because of network lag on deleting from a list. This happens only under heavy load, i.e. multiple web requests per second. My app has multiple services running. In one of the services messages will be deleted from projects.

This is the delete method in Project to remove the message first from the list and then the model itself.

    def deletemessage(m)
        messages.delete(m)  # delete from list
        m.delete   # delete model
    end

After deleting a message I get NoMethod errors from my web processes, which try to create a json feed from the messages and expect all attributes to exist.

I can see that when loading the messages with

@p = Project[4711]
@msgs = p.messages.to_a
# create json from @msgs array

the deleted messages are still inside the array, but without any attributes.

#<Message:0x007f95acf71068 @attributes={:score=>0}, @_memo={}, @id="1247196">

So I suspect Ohm got an "old" list of ids and tried to load the message, which failed. Nevertheless the empty message is inside the returned array. After a few seconds the web requests are "recovering" and now get the correct data without the deleted message. (I do not get the error, when I just delete the reference from the list and do not delete the corresponding message itself, which obviously it not what I want)

Any ideas why this is happening and what would be the best approach to solve this?

Thanks Christian

soveran commented 6 years ago

@chriso0710 Thanks for reporting this. There's a synchronization mechanism in place but it's only a mutex for same-process concurrency. I'll work on this and let you know if I find any improvement.