cloudflare / workers-sdk

โ›…๏ธ Home to Wrangler, the CLI for Cloudflare Workersยฎ
https://developers.cloudflare.com/workers/
Apache License 2.0
2.69k stars 706 forks source link

๐Ÿ› BUG: Unable to use D1 binding with local pages dev #1630

Closed Cyb3r-Jak3 closed 2 years ago

Cyb3r-Jak3 commented 2 years ago

What version of Wrangler are you using?

0.0.0-8fac577c (npx wrangler@d1 --version)

What operating system are you using?

Windows

Describe the Bug

When attempting to use D1 locally for Pages function following the docs I get the following error

F:\git\workers\car-api [D1 +6 ~5 -4 !]> npx wrangler@d1 pages dev .\public\ --d1=DB
๐Ÿšง 'wrangler pages <command>' is a beta command. Please report any issues to https://github.com/cloudflare/wrangler2/issues/new/choose
Compiling worker to "C:\Users\Jake\AppData\Local\Temp\functionsWorker-0.30216710979718786.js"...
Compiled Worker successfully.
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎโ”‚ [b] open a browser, [d] open Devtools, [c] clear console, [x] to exit                                                                                                                             โ”‚โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏDebugger listening on ws://localhost:9229/58d15b5f-751b-4df9-b4b6-8e50da2732a6
Debugger listening on ws://localhost:9229/58d15b5f-751b-4df9-b4b6-8e50da2732a6
For help, see: https://nodejs.org/en/docs/inspector
[pages:wrn] Service bindings are experimental. There may be breaking changes in the future.
[NPXI] better-sqlite3 not available locally. Attempting to use npx to install temporarily.
[pages:inf] Updated `Request.cf` object cache!
[NPXI] Installing... (npx -y -p better-sqlite3@^7.5.3)
[pages:err] Error: npx-import failed for better-sqlite3 with message:
    Failed to find temporary install directory. Looking for paths matching '/.npm/_npx/' in:
[""]

You should install better-sqlite3 locally:
    npm install --save-dev better-sqlite3@^7.5.3

    at npxImport (F:\git\workers\car-api\node_modules\node_modules\npx-import\lib\index.js:31:19)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at MemoryStorage.getSqliteDatabase (F:\git\workers\car-api\node_modules\@miniflare\storage-memory\src\local.ts:124:33)
    at BetaDatabase.init (F:\git\workers\car-api\node_modules\@miniflare\d1\src\database.ts:140:16)
    at D1Plugin.getBetaDatabase (F:\git\workers\car-api\node_modules\@miniflare\d1\src\plugin.ts:50:5)
    at D1Plugin.setup (F:\git\workers\car-api\node_modules\@miniflare\d1\src\plugin.ts:58:28)
    at EventTarget.#runSetup (F:\git\workers\car-api\node_modules\@miniflare\core\src\index.ts:320:20)
    at EventTarget.#init (F:\git\workers\car-api\node_modules\@miniflare\core\src\index.ts:452:7)
    at EventTarget.getPlugins (F:\git\workers\car-api\node_modules\@miniflare\core\src\index.ts:985:5

After installing and retrying the command, I get a new error (npxImport)

 F:\git\workers\car-api [D1 +6 ~5 -4 !]> npx wrangler@d1 pages dev .\public\ --d1=DB 
๐Ÿšง 'wrangler pages <command>' is a beta command. Please report any issues to https://github.com/cloudflare/wrangler2/issues/new/choose
Compiling worker to "C:\Users\Jake\AppData\Local\Temp\functionsWorker-0.622393753708625.js"...
Compiled Worker successfully.
โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎโ”‚ [b] open a browser, [d] open Devtools, [c] clear console, [x] to exit                                                                                                                             โ”‚โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏDebugger listening on ws://localhost:9229/5b5fd145-7ae5-44d9-9702-6be85fe17644
Debugger listening on ws://localhost:9229/5b5fd145-7ae5-44d9-9702-6be85fe17644
For help, see: https://nodejs.org/en/docs/inspector
[pages:wrn] Service bindings are experimental. There may be breaking changes in the future.
[pages:err] Error: You must call npxImport for a package before calling npxResolve. Got: better-sqlite3
    at npxResolve (F:\git\workers\car-api\node_modules\node_modules\npx-import\lib\index.js:48:15)
    at MemoryStorage.getSqliteDatabase (F:\git\workers\car-api\node_modules\@miniflare\storage-memory\src\local.ts:129:9)
    at BetaDatabase.init (F:\git\workers\car-api\node_modules\@miniflare\d1\src\database.ts:140:16)
    at D1Plugin.getBetaDatabase (F:\git\workers\car-api\node_modules\@miniflare\d1\src\plugin.ts:50:5)
    at D1Plugin.setup (F:\git\workers\car-api\node_modules\@miniflare\d1\src\plugin.ts:58:28)
    at EventTarget.#runSetup (F:\git\workers\car-api\node_modules\@miniflare\core\src\index.ts:320:20)
    at EventTarget.#init (F:\git\workers\car-api\node_modules\@miniflare\core\src\index.ts:452:7)
    at EventTarget.getPlugins (F:\git\workers\car-api\node_modules\@miniflare\core\src\index.ts:985:5)
    at createServer (F:\git\workers\car-api\node_modules\@miniflare\http-server\src\index.ts:355:19)
    at startServer (F:\git\workers\car-api\node_modules\@miniflare\http-server\src\index.ts:456:18)
[pages:inf] Updated `Request.cf` object cache!

For Windows, wrangler@d1 seems to be broken entirely. If I run npx wrangler@d1 pages dev .\public\ I get the same out as the first error (without better-sqlite3 installed). I tested with Linux using a dev container in VS Code and it appears that it will always fail when better-sqlite3 is installed via NPM. If I use npx wrangler@d1 pages dev public/ --d1=DB then it works in the container, but if I install better-sqlite3 then the command fails with the npxImport error.

philholden commented 2 years ago

Same error on Mac this is trying to add a D1 store to:

https://github.com/cloudflare/miniflare-typescript-esbuild-jest

Running custom build: node build.js
Your worker has access to the following bindings:
- Durable Objects:
  - COUNTER: Counter
- D1 Databases:
  - DB: db ()
โŽ” Starting a local server...

โ”‚ [b] open a browser, [d] open Devtools, [l] turn off local mode, [c] clear console, [x] to exit                                                                                                             
Debugger listening on ws://localhost:9229/555a0511-1988-4bef-a2ba-72f9d701ab0f
Debugger listening on ws://localhost:9229/555a0511-1988-4bef-a2ba-72f9d701ab0f
For help, see: https://nodejs.org/en/docs/inspector
[mf:err] Error: You must call npxImport for a package before calling npxResolve. Got: better-sqlite3
    at npxResolve (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/wrangler/node_modules/node_modules/npx-import/lib/index.js:48:15)
    at FileStorage.getSqliteDatabase (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/wrangler/node_modules/@miniflare/storage-file/src/index.ts:117:9)
    at BetaDatabase.init (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/@miniflare/d1/src/database.ts:140:16)
    at D1Plugin.getBetaDatabase (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/@miniflare/d1/src/plugin.ts:50:5)
    at D1Plugin.setup (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/@miniflare/d1/src/plugin.ts:58:28)
    at Miniflare.#runSetup (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/wrangler/node_modules/@miniflare/core/src/index.ts:320:20)
    at Miniflare.#init (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/wrangler/node_modules/@miniflare/core/src/index.ts:452:7)
    at Miniflare.getPlugins (/Users/philipholden/personal/gotell/miniflare-typescript-esbuild-jest/node_modules/wrangler/node_modules/@miniflare/core/src/index.ts:985:5)
โœ˜ [ERROR] Miniflare process exited with code 1
name = "miniflare-typescript-esbuild-jest"
main = "dist/index.mjs"

account_id = ""
workers_dev = true
# route = ""

compatibility_date = "2022-05-28"
compatibility_flags = []

[durable_objects]
bindings = [
  { name = "COUNTER", class_name = "Counter" }
]

[[migrations]]
tag = "v1"
new_classes = ["Counter"]

[build]
command = "node build.js"

[miniflare]
kv_persist = true
cache_persist = true
durable_objects_persist = true
https = true

[[ d1_databases ]]
binding = "DB"
database_name = "db"
database_id = ""
{
  "name": "miniflare-typescript-esbuild-jest",
  "version": "1.0.0",
  "description": "Example project using Miniflare, TypeScript, esbuild and Jest",
  "type": "module",
  "module": "./dist/index.mjs",
  "scripts": {
    "d1": "wrangler dev ./dist/index.mjs --local --experimental-enable-local-persistence",
    "build": "node build.js",
    "wrangdev": "wrangler dev --local-protocol https --experimental-enable-local-persistence --local",
    "dev": "miniflare --live-reload --debug --modules dist/index.mjs",
    "dev:remote": "wrangler dev",
    "test": "npm run build && node --experimental-vm-modules --no-warnings node_modules/jest/bin/jest.js",
    "types:check": "tsc && tsc -p test/tsconfig.json",
    "deploy": "wrangler publish"
  },
  "keywords": [],
  "author": "",
  "license": "MIT",
  "devDependencies": {
    "@cloudflare/d1": "^1.4.1",
    "@cloudflare/workers-types": "^3.11.0",
    "@types/jest": "^27.5.1",
    "better-sqlite3": "^7.6.2",
    "esbuild": "^0.14.41",
    "itty-router": "^2.6.1",
    "jest": "^28.1.0",
    "jest-environment-miniflare": "^2.6.0",
    "miniflare": "^2.6.0-d1.3",
    "prettier": "^2.6.2",
    "ts-jest": "^28.0.3",
    "typescript": "^4.7.2",
    "wrangler": "^0.0.0-8fac577c"
  }
}
philholden commented 2 years ago

I was running npm run d1 note the node_modules/node_modules in the first line of mf:err

geelen commented 2 years ago

This looks like the same bug as what I just fixed with https://github.com/geelen/npx-import/pull/4

I've pushed a new wrangler@d1 v0.0.0-205f5496, which uses miniflare@2.6.0-d1.4 and npx-import@1.0.2, can you try updating and let me know if the same error occurs?

I'll dig into it more if so!

geelen commented 2 years ago

@Cyb3r-Jak3 I think the first error is actually related to https://github.com/geelen/npx-import not being tested on windows yet. I've raised https://github.com/geelen/npx-import/issues/5 to track it. But it sounds like the second case, when you install better-sqlite3 locally, is now fixed...

Cyb3r-Jak3 commented 2 years ago

@geelen Confirmed second case is fixed and still errors with first case. Let me know if I can help with https://github.com/geelen/npx-import/issues/5

philholden commented 2 years ago

Confirming fixed on mac when better-sqlite installed

WalshyDev commented 2 years ago

Hey,

This should now be resolved, let us know if it is not!