search

tndrle / node-sqlite3-wasm

WebAssembly port of SQLite3 for Node.js with file system access
MIT License
55 stars 8 forks source link

Strings and arrays passed via heap instead of stack #33

Closed tndrle closed 10 months ago

tndrle commented 10 months ago

Fixes #30

Currently, node-sqlite3-wasm passes strings and arrays to C functions via the stack. However, the stack cannot grow, and therefore, the string and array sizes are limited. Passing strings and arrays via the heap should solve this.

mzoliker commented 9 months ago

Hi, sorry if this is a stupid question, but what is the point to _malloc, then call sqlite api with SQLITE_TRANSIENT and then _free the string? As SQLITE_TRANSIENT already tells sqlite to keep its own private copy of the string, aren’t you actually allocating 2 copies of the same string instead of simply letting sqlite do the job? Am I missing something? https://www.sqlite.org/c3ref/c_static.html Thanks a lot for your feedback! Maurice

tndrle commented 9 months ago

Hi, thanks for the comment. In my opinion, there are two options: SQLITE_STATIC and SQLITE_TRANSIENT. With SQLITE_STATIC, I (i.e. the API side of node-sqlite3-wasm) have to guarantee that the string is available at the same location in memory in the future, e.g. until the query is actually executed. With SQLITE_TRANSIENT, SQLite creates its own copy of the string and I don't need to hold it in memory any longer. That means SQLITE_TRANSIENT is easier to use because SQLITE_STATIC requires careful memory management. To be on the safe side, I use SQLITE_TRANSIENT for the moment. Copying the string to the heap with _malloc is necessary because emscripten doesn't grow the WASM stack and, thus, cannot hold large strings (see #30).