mfp / ocaml-sqlexpr

Minimalistic syntax extension for type-safe, convenient execution of SQL statements.
Other
86 stars 17 forks source link

Excessive GC usage when creating many (uncached) statements or DB handles (e.g. sqlinit) #17

Closed mfp closed 7 years ago

mfp commented 8 years ago

sqlite3-ocaml's statements and database handles are allocated with caml_alloc_final and caml_alloc_custom, with used=1,max=100 parameters, see https://github.com/mmottl/sqlite3-ocaml/blob/master/lib/sqlite3_stubs.c#L775

The obvious workaround for the former is "use sqlc instead of sql".

However, sqlinit is not cached, so if you're e.g. executing many "CREATE TABLE IF NOT EXISTS ..." to initialize the schema on DB open, you're getting a full major GC run every 100 statements. This hurts if the program is opening many DBs.

I'm pondering whether to change sqlinit to cache the statements, or introduce a caching version (sqlcinit).

Can anybody think of a scenario where caching the statements would be a bad thing?

Indeed, it might even make sense to make sql behave like sqlc regarding caching. (There's also the code hoisting thing, so there's a small justification not to deprecate one of them, but the effect is comparatively minor in both directions -- neither hoisting all the time nor never doing it should really matter performance-wise).