search

paul-rouse / mysql-simple

A mid-level client library for the MySQL database, intended to be fast and easy to use.
Other
91 stars 35 forks source link

Properly use new concurrency functions in mysql #34

Open 3noch opened 7 years ago

3noch commented 7 years ago

Now that mysql finally has the proper concurrency fixes, downstream libraries need to start using the concurrency-related init functions (initLibrary and initThread) to get full, safe concurrency from MySQL at the C level.

The changes are here: https://github.com/paul-rouse/mysql/pull/15/files

Making a safe, easy-to-use Haskell API that includes these may prove challenging however.

paul-rouse commented 7 years ago

It is certainly a challenge! When I fixed the Yesod scaffolding to use the new functions, I had to pass several settings to the warp server to control, and hook into, the way the server handles threading - and I had to dive into the warp code to check that the behaviour was really what I wanted. (Details at http://www.yesodweb.com/blog/2016/11/use-mysql-safely-in-yesod). The point is that the changes cut across the application's threading model, which may well be implemented in other libraries, and it is hard to see how mysql-simple could hide this in any useful way.

I'd love to see some proposals!

3noch commented 7 years ago

I can't recall if the C library wants you to use a connection within the same thread at all times (i.e. if it's safe to use the same connection in different threads over time). I do recall thinking over this problem years ago when I first discovered the issue in mysql. At the time the best idea I came up with was to use some globals (it could be done with a custom monad, but that would be a drastic API change) to keep track of whether or not the library had been initialized and which capabilities (OS threads) had been initialized. Then each call to the DB from mysql-simple would ensure that the proper initialization had been completed. This approach would obviously add some overhead which could be mitigated by introducing unsafe* variants of many operations (i.e. offload the responsibility to the library user if speed is a problem). And/or one could introduce a custom monad [transformer] that would keep track of this state in a more optimal way.

One benefit of a monad would be that it could include safe handling of connection pooling when using transactions, which would also help make the story for concurrency better.