search

makerplane / FIX-Gateway

Flight Information eXchange Gateway
GNU General Public License v2.0
10 stars 22 forks source link

Database Locking #9

Closed birkelbach closed 5 years ago

birkelbach commented 8 years ago

We've never implemented any locking on the database. Most of the plugins operate in their own thread and all access the database at some level. It seems to be working well enough. I know in Python a lot of things are atomic that are not in other languages. I can't really convince myself that we need it and it's seems to be working really well, but it goes against the grain on multithreaded programing to not lock common resources. Comments?

neil-d95 commented 8 years ago

I was talking with a coworker about this if we are talking about a multiple read, single write for a value. We might seen unusual thing in the data if it's reading while writing where as if we use a lock for the write process we can pull the most current data as well as mitigate odd data.

birkelbach commented 8 years ago

We are actually doing multiple writes and multiple reads (or at least it's possible to do so). What I understand is that any read or write of a Python builtin type is guaranteed to be atomic. Also the Python interpreter has the GIL so there is some locking going on inside the interpreter itself. I don't know to what extent that helps. My worry on locking is performance, but we may not have a performance problem. Once we start throwing a bunch more data at this thing we'll see. I should probably write some kind of test and see if I can make it fail.

neil-d95 commented 8 years ago

Yeah that sounds like a this might need to stay out there till we get a handle on how the GIL locking holds up.

On 06/15/16 07:30, Phil Birkelbach wrote:

We are actually doing multiple writes and multiple reads (or at least it's possible to do so). What I understand is that any read or write of a Python builtin type is guaranteed to be atomic. Also the Python interpreter has the GIL so there is some locking going on inside the interpreter itself. I don't know to what extent that helps. My worry on locking is performance, but we may not have a performance problem. Once we start throwing a bunch more data at this thing we'll see. I should probably write some kind of test and see if I can make it fail.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/makerplane/FIX-Gateway/issues/9#issuecomment-226161010, or mute the thread https://github.com/notifications/unsubscribe/AEjkWW6jLEMVB3o0QTbpLvnnUEOX7pWvks5qL-JQgaJpZM4I1Z6A.

birkelbach commented 8 years ago

If I was seeing any problems or weird data I'd throw some locking at it in a heartbeat, but it all seems to be working fine. My embedded and C programming experience says that this should not be... but Python is a different beast.

birkelbach commented 5 years ago

I've done a bit more research on this issue. It seems that dictionary accesses are indeed atomic in CPython but it's not specified behavior, rather it's a side effect of the implementation. It would be best if database locking was added.