Could not find how to perform per-thread initialization e.g. for sqlite

[cpbotha]

In sqlite3's MULTITHREAD mode, one has to use a separate database connection for each thread.

I've gone through the jester and httpbeast source, but could not find where / how to perform per-thread init, for example to open a sqlite3 connection for each thread.

Unless I have misunderstood, I need a { .threadVar. } db connection variable, but then have each thread init that separately. I don't see any places where I could easily hook this up, but I would love to be corrected!

(alternatively, one could opt to open the sqlite3 database in SERIALIZED mode to share the same connection option over threads, but the nim sqlite3 wrapper does not yet support the pre-requisite function calls (I might try to PR this at some point). Read more about sqlite3 threading modes here: https://www.sqlite.org/threadsafe.html

The default mode depends on sqlite3 compilation options. Here on my dev mac, it was compiled with SQLITE_CONFIG_THREADSAFE=2, which means a default of MULTITHREAD and not SERIALIZED. I would have to configure at runtime with either sqlite3_config() or preferably sqlite3_open_v2())

[Araq]

Is Jester even creating threads under the hood? That's news to me. If it doesn't, you're free to setup the threads on your own and setting up the SQlite connections. Right?

[cpbotha]

When the jester app is compiled with --threads:on, httpbeast's run() will spawn countProcessor() threads, see https://github.com/dom96/httpbeast/blob/master/src/httpbeast.nim#L415

It would be useful to be able to perform some sort of per-thread init from the jester side, e.g. creating a sqlite connection, amongst others.

[dom96]

It would be useful to perform per-thread init from the Nim side (I've said this multiple times now too). Otherwise every library will need to provide hooks into the thread startup which will be a PITA.

[dom96]

Anyway, you can create a proc that initializes these thread vars for you if they are nil. Just create a threadvar for your connection and a proc like:

proc getSqliteConnection() =
  if sqliteConnection.isNil:
    # Initialise
  return sqliteConnection

[Araq]

I've said this multiple times now too

And I've said it multiple times now too, been there, done that, it's more complex than it looks and encourages data races.

[dom96]

And I've said it multiple times now too, been there, done that, it's more complex than it looks and encourages data races.

Can you elaborate and explain how it encourages data races and why it is complex?

[Araq]

For example, threadpools creates the worker threads at "init" time, then later modules are initialized that call something like atThreadCreation(...) and by then it's too late, the threads are started and cannot see that they should have run the registered callbacks.