search

drizzle-team / drizzle-orm

Headless TypeScript ORM with a head. Runs on Node, Bun and Deno. Lives on the Edge and yes, it's a JavaScript ORM too 😅
https://orm.drizzle.team
Apache License 2.0
24.65k stars 650 forks source link

[BUG]:Drizzle + SQLite + Bun.JS does not work #1179

Closed otekdo closed 1 year ago

otekdo commented 1 year ago

What version of drizzle-orm are you using?

0.28.5

What version of drizzle-kit are you using?

0.19.13

Describe the Bug

It does not migrate the schema to the database file

Repository Link: https://github.com/otekdo/bunjs-drizzle-sqlite

generate command works

$ bun drizzle-kit generate:sqlite
drizzle-kit: v0.19.13
drizzle-orm: v0.28.5

No config path provided, using default 'drizzle.config.ts'
Reading config file '/home/nexus/Documents/apps/bun-drizzle-sqlite/drizzle.config.ts'
1 tables
users 8 columns 0 indexes 0 fks

[✓] Your SQL migration file ➜ src/db/migrations/0000_bizarre_morlocks.sql 🚀

push command does not works

No config path provided, using default path
Reading config file '/home/nexus/Documents/apps/bun-drizzle-sqlite/drizzle.config.ts'
drizzle-kit: v0.19.13
drizzle-orm: v0.28.5

/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/bindings/bindings.js:126
  err = new Error(
        ^

Error: Could not locate the bindings file. Tried:
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/Debug/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/Release/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/out/Debug/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/Debug/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/out/Release/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/Release/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/default/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/compiled/18.17.1/linux/x64/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/addon-build/release/install-root/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/addon-build/debug/install-root/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/addon-build/default/install-root/better_sqlite3.node
 → /home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/lib/binding/node-v108-linux-x64/better_sqlite3.node
    at bindings (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/bindings/bindings.js:126:9)
    at new Database (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/lib/database.js:48:64)
    at _createClient (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/drizzle-kit/index.cjs:43506:14)
    at _createClient4 (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/drizzle-kit/index.cjs:51839:12)
    at createClient (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/drizzle-kit/index.cjs:51834:10)
    at connectToSQLite (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/drizzle-kit/index.cjs:51888:13)
    at Command.<anonymous> (/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/drizzle-kit/index.cjs:53433:28) {
  tries: [
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/Debug/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/Release/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/out/Debug/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/Debug/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/out/Release/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/Release/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/default/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/compiled/18.17.1/linux/x64/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/addon-build/release/install-root/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/addon-build/debug/install-root/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/addon-build/default/install-root/better_sqlite3.node',
    '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/lib/binding/node-v108-linux-x64/better_sqlite3.node'
  ]
}

Node.js v18.17.1
error: "drizzle-kit" exited with code 1 (SIGHUP)
error: script "push" exited with code 1 (SIGHUP)

It does not matter which driver is set on drizzle config file, issue still happens libsql or better-sqlite

Expected behavior

Generate the db file with its table created and then show empty row of users when running Bun web server

Environment & setup

Repository: https://github.com/otekdo/bunjs-drizzle-sqlite Bun: v0.8.1 drizzle-kit: v0.19.13 drizzle-orm: v0.28.5

kansson commented 1 year ago

You need to install better-sqlite3 and run it's build-debug script like this.

bun install --dev better-sqlite3
bun run --cwd node_modules/better-sqlite3 build-debug
otekdo commented 1 year ago

@hanssonduck Second command did not work First command

bun install --dev better-sqlite3
bun add v0.8.1
installed better-sqlite3@8.6.0
[547.00ms] done

Second command

bun run --cwd node_modules/better-sqlite3 build-debug
$ node-gyp rebuild --debug
/usr/bin/bash: line 1: node-gyp: command not found
error: script "build-debug" exited with code 127

I also have NodeJs installed, v.18.17.1

kansson commented 1 year ago

@hanssonduck Second command did not work First command

bun install --dev better-sqlite3
bun add v0.8.1
installed better-sqlite3@8.6.0
[547.00ms] done

Second command

bun run --cwd node_modules/better-sqlite3 build-debug
$ node-gyp rebuild --debug
/usr/bin/bash: line 1: node-gyp: command not found
error: script "build-debug" exited with code 127

I also have NodeJs installed, v.18.17.1

Install it with bun add --global node-gyp and try again.

otekdo commented 1 year ago

Sharing progress Build debug results

bun run --cwd node_modules/better-sqlite3 build-debug
$ node-gyp rebuild --debug
gyp info it worked if it ends with ok
gyp info using node-gyp@9.4.0
gyp info using node@18.17.1 | linux | x64
gyp info find Python using Python version 3.10.12 found at "/usr/bin/python3"
gyp http GET https://nodejs.org/download/release/v18.17.1/node-v18.17.1-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v18.17.1/node-v18.17.1-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v18.17.1/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v18.17.1/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/home/nexus/.bun/install/global/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/nexus/.bun/install/global/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/nexus/.cache/node-gyp/18.17.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/nexus/.cache/node-gyp/18.17.1',
gyp info spawn args   '-Dnode_gyp_dir=/home/nexus/.bun/install/global/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/nexus/.cache/node-gyp/18.17.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Debug', '-C', 'build' ]
make: Entering directory '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build'
  TOUCH ba23eeee118cd63e16015df367567cb043fed872.intermediate
  ACTION deps_sqlite3_gyp_locate_sqlite3_target_copy_builtin_sqlite3 ba23eeee118cd63e16015df367567cb043fed872.intermediate
  TOUCH Debug/obj.target/deps/locate_sqlite3.stamp
  CC(target) Debug/obj.target/sqlite3/gen/sqlite3/sqlite3.o
rm -f Debug/obj.target/deps/sqlite3.a Debug/obj.target/deps/sqlite3.a.ar-file-list; mkdir -p `dirname Debug/obj.target/deps/sqlite3.a`
ar crs Debug/obj.target/deps/sqlite3.a @Debug/obj.target/deps/sqlite3.a.ar-file-list
  COPY Debug/sqlite3.a
  CXX(target) Debug/obj.target/better_sqlite3/src/better_sqlite3.o
In file included from ./src/better_sqlite3.lzz:11,
                 from ../src/better_sqlite3.cpp:4:
/home/nexus/.cache/node-gyp/18.17.1/include/node/node.h:1034:7: warning: cast between incompatible function types from ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, v8::Local<v8::Context>)’ to ‘node::addon_context_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, v8::Local<v8::Context>, void*)’} [-Wcast-function-type]
 1034 |       (node::addon_context_register_func) (regfunc),                  \
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/nexus/.cache/node-gyp/18.17.1/include/node/node.h:1052:3: note: in expansion of macro ‘NODE_MODULE_CONTEXT_AWARE_X’
 1052 |   NODE_MODULE_CONTEXT_AWARE_X(modname, regfunc, NULL, 0)
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/nexus/.cache/node-gyp/18.17.1/include/node/node.h:1083:3: note: in expansion of macro ‘NODE_MODULE_CONTEXT_AWARE’
 1083 |   NODE_MODULE_CONTEXT_AWARE(NODE_GYP_MODULE_NAME,                     \
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~
./src/better_sqlite3.lzz:67:1: note: in expansion of macro ‘NODE_MODULE_INIT’
./src/util/data.lzz: In function ‘v8::Local<v8::Value> Data::GetValueJS(v8::Isolate*, sqlite3_stmt*, int, bool)’:
./src/util/data.lzz:73:92: warning: this statement may fall through [-Wimplicit-fallthrough=]
./src/util/data.lzz:73:197: note: here
./src/util/data.lzz: In function ‘v8::Local<v8::Value> Data::GetValueJS(v8::Isolate*, sqlite3_value*, bool)’:
./src/util/data.lzz:77:81: warning: this statement may fall through [-Wimplicit-fallthrough=]
./src/util/data.lzz:77:175: note: here
  SOLINK_MODULE(target) Debug/obj.target/better_sqlite3.node
  COPY Debug/better_sqlite3.node
  CC(target) Debug/obj.target/test_extension/deps/test_extension.o
  SOLINK_MODULE(target) Debug/obj.target/test_extension.node
  COPY Debug/test_extension.node
rm ba23eeee118cd63e16015df367567cb043fed872.intermediate
make: Leaving directory '/home/nexus/Documents/apps/bun-drizzle-sqlite/node_modules/better-sqlite3/build'
gyp info ok
otekdo commented 1 year ago

Great, it worked! @hanssonduck a lot of thanks!

bun run push
$ bun drizzle-kit push:sqlite
No config path provided, using default path
Reading config file '/home/nexus/Documents/apps/bun-drizzle-sqlite/drizzle.config.ts'
drizzle-kit: v0.19.13
drizzle-orm: v0.28.5

[✓] Changes applied

image

otekdo commented 1 year ago

Looks like the setup documentation on drizzle should be updated https://orm.drizzle.team/docs/installation-and-db-connection/sqlite/bun That is the page i was reading/following

I was not aware that i needed to run the command @hanssonduck recommended, by the time i will update my README.md file on my repo

kansson commented 1 year ago

Looks like the setup documentation on drizzle should be updated https://orm.drizzle.team/docs/installation-and-db-connection/sqlite/bun That is the page i was reading/following

I was not aware that i needed to run the command @hanssonduck recommended, by the time i will update my README.md file on my repo

This should be fixed when Drizzle Kit drop Node as a dependency.

mateusavila commented 1 year ago

I found the solution to my problem, but the documentation is not updated yet.

jrey8343 commented 10 months ago

Hi guys - I found a more elegant solution to this problem

better-sqlite3 wants to run post-install scripts which are not trusted by Bun by default.

As per Bun's documentation you need to add the following line to your package.json

"trustedDependencies": [ "better-sqlite3" ],

You can then delete your node_modules folder and bun.lockb files Re-install with bun install and you are good to go!

Here is the link to the docs https://bun.sh/guides/install/trusted

pekeler commented 6 months ago

When using Bun, you don't need an SQLite library such as better-sqlite3 anymore. So requiring better-sqlite3 just to run migrations seems wrong.

https://bun.sh/guides/ecosystem/drizzle has a nicer workaround for this issue.

I vote to re-open this issue. And would suggest allowing 'bun:sqlite' as a possible value for the dialect setting in drizzle.config.ts, then using that by drizzle-kit to figure out what implementation to use.