search

sveltejs / kit

web development, streamlined
https://svelte.dev/docs/kit
MIT License
18.71k stars 1.94k forks source link

importing sqlite3 fails in dev mode #3118

Closed lovasoa closed 2 years ago

lovasoa commented 2 years ago

Describe the bug

A recent update to sveltekit broke the usage of sqlite3 in server-side code.

When using npm's sqlite3 module in server-side code and running the code in development mode, the import fails with

Cannot find module 'node_modules/sqlite3/package.json' imported from 'node_modules/node-pre-gyp/lib/pre-binding.js'
    at viteResolve (node_modules/vite/dist/node/chunks/dep-4a9cff06.js:67808:25)

Reproduction

use sqlite3 in a svelte hook, and run the code with svelte-kit dev

Reproducible in https://github.com/lovasoa/sanipasse/tree/5f4a3d5177c88462b2bee2c029eeb7534155cdb2

Logs

Error: Cannot find module '/home/olojkine/dev/sanipasse/node_modules/sqlite3/package.json' imported from '/home/olojkine/dev/sanipasse/node_modules/node-pre-gyp/lib/pre-binding.js'
    at viteResolve (/home/olojkine/dev/sanipasse/node_modules/vite/dist/node/chunks/dep-4a9cff06.js:67808:25)
    at Function._resolveFilename (/home/olojkine/dev/sanipasse/node_modules/vite/dist/node/chunks/dep-4a9cff06.js:67822:20)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.exports.find (/home/olojkine/dev/sanipasse/node_modules/node-pre-gyp/lib/pre-binding.js:20:23)
    at Object.<anonymous> (/home/olojkine/dev/sanipasse/node_modules/sqlite3/lib/sqlite3-binding.js:3:27)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)

System Info

System:
    OS: Linux 5.15 Fedora Linux 35 (Workstation Edition)
    CPU: (8) x64 Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
    Memory: 1.18 GB / 15.28 GB
    Container: Yes
    Shell: 5.1.8 - /bin/bash
  Binaries:
    Node: 16.13.0 - /usr/bin/node
    Yarn: 1.22.10 - /usr/bin/yarn
    npm: 8.1.0 - /usr/bin/npm
  Browsers:
    Firefox: 95.0
  npmPackages:
    @sveltejs/adapter-node: next => 1.0.0-next.56 
    @sveltejs/adapter-static: next => 1.0.0-next.22 
    @sveltejs/kit: next => 1.0.0-next.202 
    svelte: ^3.38.3 => 3.44.3

Severity

blocking an upgrade

Additional Information

No response

bluwy commented 2 years ago

The core issue might be the same as #3053, but the stack trace looks new to me. I'm starting to think that Vite shouldn't interfere with Node's internal stuff, as you've found in vitejs/vite#3951. I haven't got the bottom of fixing that yet, but I'm feeling we should revert that.

bluwy commented 2 years ago

@lovasoa Can you provide a smaller repro for sqlite3 only? There's a lot going on in your project it was tricky to track it down.

lovasoa commented 2 years ago

@bluwy : I tried, but the bug is quite hard to reproduce in a minimal context. It seems to be dependent on the order of compilation of different modules. To help you with debugging, I created a new branch in my repo with a test endpoint:

https://github.com/lovasoa/sanipasse/tree/reproduction-sveltekit-bug-3118

https://github.com/lovasoa/sanipasse/blob/reproduction-sveltekit-bug-3118/src/routes/api/test.ts

If you load this endpoint first immediately after starting the dev server, then the code is compiled correctly and the application will work correctly for the entire lifetime of the dev server.

If you first load an other page (/ for instance), then the application will be compiled incorrectly, and will fail every time $lib/database is imported.

The following fails:

The following works:

lovasoa commented 2 years ago

@bluwy Okay, knowing this load order issue, I could reproduce the issue in a minimal context :

https://github.com/lovasoa/sveltekit-bug-3118/

ceifa commented 2 years ago

@bluwy Okay, knowing this load order issue, I could reproduce the issue in a minimal context

Did you undertand how exactly this happens? I'm having the same problem with marked/package.json on svelte-markdown.

lovasoa commented 2 years ago

No, I didn't dig deep enough to understand how this happens. Sorry !

bluwy commented 2 years ago

@lovasoa An update, it's the weekdays again so I'm a bit busy, but I'll try to take a look when I have the time. But big thanks for the smaller reproduction, that would help a lot when debugging.

bluwy commented 2 years ago

I've sent a fix at https://github.com/vitejs/vite/pull/6488

bluwy commented 2 years ago

Closing this as vitejs/vite#6488 has been merged. Once that is released, you can update SvelteKit's Vite version to that new version.

ceifa commented 2 years ago

Closing this as vitejs/vite#6488 has been merged. Once that is released, you can update SvelteKit's Vite version to that new version.

Hey, the problem still persists in the latest version of kit. I think it's fair to close this issue only after vitejs version is updated on kit, since it's not a peer dependency.

bluwy commented 2 years ago

@ceifa Vite 2.8.0-beta.3 is out with the fix. You can force to that version with npm overrides, yarn resolutions, or pnpm overrides. But if the beta has other issues, perhaps I can check with the Vite team to release the fix under 2.7.x, but so far vite-ecosystem-ci for svelte is passing.

lubiah commented 2 years ago

@ceifa Vite 2.8.0-beta.3 is out with the fix. You can force to that version with npm overrides, yarn resolutions, or pnpm overrides. But if the beta has other issues, perhaps I can check with the Vite team to release the fix under 2.7.x, but so far vite-ecosystem-ci for svelte is passing.

How would you do this in your package.json?

uc-anr commented 2 years ago

@ceifa Vite 2.8.0-beta.3 is out with the fix. You can force to that version with npm overrides, yarn resolutions, or pnpm overrides. But if the beta has other issues, perhaps I can check with the Vite team to release the fix under 2.7.x, but so far vite-ecosystem-ci for svelte is passing.

How would you do this in your package.json?

bluwy commented 2 years ago

@kudadam @uc-anr I've already linked the relevant docs. Here's the answer:

{
  "name": "blabla",
  "overrides": {
    "vite": "^2.8.0-beta.5"
  }
}
lubiah commented 2 years ago

@kudadam @uc-anr I've already linked the relevant docs. Here's the answer:

{
  "name": "blabla",
  "overrides": {
    "vite": "^2.8.0-beta.5"
  }
}

I did the same thing but the error continues to pop up

uc-anr commented 2 years ago

@kudadam @uc-anr I've already linked the relevant docs. Here's the answer:

{
  "name": "blabla",
  "overrides": {
    "vite": "^2.8.0-beta.5"
  }
}

Can you please clearly share some doc where it is used as we are getting this issues continuously after that also.

bluwy commented 2 years ago

Did y'all run npm i after updating package.json as well? Otherwise please provide a minimal reproduction repo / stackblitz so I can check it out. Docs: https://docs.npmjs.com/cli/v8/configuring-npm/package-json#overrides. Use it on a package.json

uc-anr commented 2 years ago

Did y'all run npm i after updating package.json as well? Otherwise please provide a minimal reproduction repo / stackblitz so I can check it out. Docs: https://docs.npmjs.com/cli/v8/configuring-npm/package-json#overrides. Use it on a package.json

{ "name": "ucertify", "overrides": { "vite": "^2.8.0-beta.5" }, "version": "0.0.1", "description": "Interactive Courses & Labs -uCertify", "scripts": { "dev": "svelte-kit dev", "test": "jest test", "build": "svelte-kit build", "preview": "svelte-kit preview", "start": "node build", "check": "svelte-check --tsconfig ./tsconfig.json", "check:watch": "svelte-check --tsconfig ./tsconfig.json --watch", "format": "prettier --write ./src/*/.{js,svelte,html,ts}" }, "devDependencies": { "@babel/core": "^7.15.0", "@babel/plugin-syntax-dynamic-import": "^7.0.0", "@babel/plugin-transform-runtime": "^7.0.0", "@babel/preset-env": "^7.15.0", "@babel/runtime": "^7.0.0", "@sveltejs/adapter-static": "^1.0.0-next.13", "@sveltejs/kit": "next", "@testing-library/svelte": "^3.0.3", "@types/jest": "^27.0.0", "@types/node": "^16.3.2", "@typescript-eslint/eslint-plugin": "^5.10.2", "@typescript-eslint/parser": "^5.10.2", "babel-jest": "^26.6.3", "eslint": "^7.32.0", "eslint-config-prettier": "^8.1.0", "eslint-config-standard": "^16.0.3", "eslint-plugin-import": "^2.25.4", "eslint-plugin-node": "^11.1.0", "eslint-plugin-promise": "^5.2.0", "eslint-plugin-svelte3": "^3.2.0", "nyc": "^15.1.0", "prettier": "~2.2.1", "prettier-plugin-svelte": "^2.2.0", "rollup-plugin-copy": "^3.4.0", "svelte": "^3.41.0", "svelte-check": "^2.0.0", "svelte-jester": "^2.0.1", "svelte-preprocess": "^4.0.0", "tslib": "^2.0.0", "typescript": "^4.0.0", "vite-plugin-compression": "^0.3.6" }, "type": "module", "dependencies": { "@fontsource/fira-mono": "^4.2.2", "@lukeed/uuid": "^2.0.0", "@sentry/browser": "^6.16.1", "@sentry/tracing": "^6.16.1", "@sveltejs/adapter-node": "^1.0.0-next.33", "axios": "^0.21.1", "bowser": "^2.11.0", "buffer": "^6.0.3", "cookie": "^0.4.1", "crypto-js": "^4.1.1", "highcharts": "^9.1.2", "html-entities": "^2.3.2", "idle-session-timeout": "^1.0.1", "jest": "^27.0.6", "lodash": "^4.17.21", "moment": "^2.29.1", "scenejs": "^1.5.0", "screenfull": "^5.1.0", "svelte-calendar": "^2.0.4", "svelte-file-dropzone": "^0.0.15", "svelte-i18n": "^3.3.9", "svelte-moveable": "^0.18.0", "svelte-paginate": "^0.0.1", "svelte-progresscircle": "^0.3.0", "svelte-qrcode": "^1.0.0", "sweetalert": "^2.1.2", "tippy.js": "^6.3.1" }, "babel": { "presets": [ [ "@babel/preset-env", { "targets": { "node": "current" } } ] ], "plugins": [ "@babel/plugin-proposal-class-properties" ] } }

Above is my package.json pls check.