search

chapeldev / cdo

Chapel Data Object
Apache License 2.0
10 stars 6 forks source link

Locking contention between Chapel and libpq #23

Open buddha314 opened 6 years ago

buddha314 commented 6 years ago

@tshimanga has been working on a locking contention issue, where the threading used in libpq conflicts with Chapel's native threading methods. Or at least it appears to.

buddha314 commented 6 years ago

OH MY GOD! I spelled Kerim's name correctly without looking!

marcoscleison commented 6 years ago

I know that CDO cursors need to be improved having performance in mind. But I really want to understand this lock issue. If it is in CDO, it will easy to solve, but if it is related to libpq we need an approach to solve it.

buddha314 commented 6 years ago

This is tied to #22 Here is an example of a PG library based on non-blocking sockets: https://github.com/leafo/pgmoon/tree/master/pgmoon

@ben-albrecht One of our engineers suggested this would be much easier of Chapel had non-blocking sockets. Any opinion on that?

ronawho commented 6 years ago

Shot in the dark (I don't know anything about libpq or what you're trying to do in general) -- if the general problem is that threads created in an external library seem to be contending with Chapel threads/tasks one thing you could try is export QT_AFFINITY=no and if that doesn't help maybe also set QT_SPINCOUNT=300

qthreads (our default tasking layer) likes to assume it owns the entire machine, so it will do things like pin worker pthreads to cores, and spinwait while waiting for more tasks. We have seen cases (usually with Chapel+OpenMP interop) where that ends up just creating a lot of contention and slows things down, so disable affinity/pinning and lowering the amount of spinwaiting may help.

ben-albrecht commented 6 years ago

One of our engineers suggested this would be much easier of Chapel had non-blocking sockets.

I'll defer this one to @mppf, though he may have already addressed this in Gitter discussion.

mppf commented 6 years ago

Chapel doesn't have a Chapelriffic sockets library at all right now. It does provide thin wrappers around the C sockets functions. These can already be used to create nonblocking sockets. Of course, it would be great if these were accessible in a more Chapel friendly way.