[Bug]: SQLite3 (0.11.1) incompatible with Deno Deploy?

[dankoster]

Problem description

SQLite works great locally but fails when deploying to Deno Deploy with the following error, which hides the actual error.

Error: Failed to load SQLite3 Dynamic Library
    at https://jsr.io/@db/sqlite/0.11.1/src/ffi.ts:642:9

Steps to reproduce

1. Create a new Deno Deploy playground
2. Copy the example code from https://jsr.io/@db/sqlite or https://deno.land/x/sqlite3@0.11.1
3. Paste the example code into the Deno Deploy playground
4. Hit the "Save and Deploy" button

I made my repro public here: https://dash.deno.com/playground/old-moth-94

Expected behavior

No error when deploying.

Environment

Latest Deno Deploy (fails)

Local (works as expected):

MacBook Air 15-inch, M2, 2023, macOS 14.5
deno 1.45.2 (release, aarch64-apple-darwin)
v8 12.7.224.12
typescript 5.5.2
deployctl 1.12.0

Possible solution

Is this because --unstable-ffi is required? Is there a way to deal with this on Deno Deploy?

Possible debugging: The stack trace points to ffi.ts:642 as the source of the error. That line does include the original exception, but the deployment output is not sufficiently verbose to reveal that information. If the verbosity cannot be increased then a next course of action may be to include the SQLite code directly instead of as a dependency and modify this line to have more verbose output.

} catch (e) {
  if (e instanceof Deno.errors.PermissionDenied) {
    throw e;
  }

  throw new Error("Failed to load SQLite3 Dynamic Library", { cause: e });  //<--- ERROR HERE
}

Additional context

Fails in playground, deployctl, and github action.

[magurotuna]

Hi @dankoster, unfortunately FFI is not supported on Deno Deploy, so any library that depends on this capability doesn't work.

[dankoster]

@magurotuna Thanks for the reply! Do you know if FFI on the roadmap for Deno Deploy? Will it be possible to use SQLite in the future?

[magurotuna]

Sorry to say that it's not on the roadmap. Deno Deploy aims to be secure and it achieves this by utilizing V8 isolation and Deno's permission model. However, once we allow FFI, that would break the security model, or alternatively we would need to very carefully redesign the entire architecture to ensure security.

In general, in case you want to rely on something outside JS, you may want to consider wasm, which is supported on Deno Deploy. Also, we offer Deno KV that is built into Deno and Deno Deploy so you can use it right now without the need to complex setup. It is not the same as SQLite, but I believe it covers a good portion of usecases you can do with SQLite.

[dankoster]

Using a KV store would, paradoxically, over-complicate the data storage task I want to perform. I will explore the WASM approach for now and keep hosted SQL solutions in my pocket for the unlikely scenario where I need to scale up.

sqlite3 WebAssembly & JavaScript Documentation Index Connect to SQLite with the WASM-Optimized Module

@magurotuna Thank you for your help and thank you for being security conscious!

[dankoster]

@magurotuna A WASM implementation is fairly useless on deno deploy if we don't have permission to write to the database file. Are ANY permissions configurable on deno deploy?

PermissionDenied - Requires write access to "./test.db", run again with the --allow-write flag

[magurotuna]

On Deno Deploy file system related operations are limited to a set of read operations, and write operations are not allowed. https://docs.deno.com/deploy/api/runtime-fs/ Basically a good mental model for Deno Deploy is a simple computational resource that can be invoked via HTTP, with very limited immutable file system attached. If you'd like to persist some data, Deno KV is the only option available within Deno Deploy, or for more general purposes consider using external storage services, like S3, GCS, etc.

[dankoster]

Ok, thanks for the clarification! :)