typicode / lowdb

Simple and fast JSON database
MIT License
21.3k stars 918 forks source link

Can't import JSONFile from "lowdb/node" import. #554

Closed Trystan-SA closed 1 year ago

Trystan-SA commented 1 year ago

When I use import {JSONFile} from 'lowdb/lib/node' it return the following error message.

[ERROR] Could not resolve "lowdb/lib/node"
    src/users.ts:2:23:
      2 │ import {JSONFile} from 'lowdb/lib/node'
        ╵                        ~~~~~~~~~~~~~~~~
  The path "./lib/node" is not exported by package "lowdb":

I'm using Esbuild, I tried to make this package external, or tried find another import path, but I can't make it work without modifying the lowdb source code. This is what I did :

index.js

export * from './adapters/Memory.js';
export * from './adapters/MemorySync.js';
export * from './Low.js';
export * from './LowSync.js';
+ export * from './node.js';

I tried to use the documentation path import {JSONFile} from 'lowdb/node' but typescript can't find the module.

typicode commented 1 year ago

Hi,

Could you please check that you're using latest lowdb version? I have it working with v5.0.5 and TypeScript in a project I'm working on.

nooptr commented 1 year ago

@typicode I also had the same problem. I use mac m1 pro. Using node 16.14.0, lowdb v5.0.5

Trystan-SA commented 1 year ago

I use the latest 5.0.5 version.

ghost commented 1 year ago

Issue happens on the lowdb/browser imports too. I can reproduce using a default Next.js project + lowdb: https://github.com/jasonbarone/lowdb-type-error-test. Node 16.15.0, M1 Pro, pnpm 7.13

typicode commented 1 year ago

Can you try setting "type": "module" in your package.json? Here's also how you can test locally:

package.json

{
  "name": "lowdb-test",
  "version": "1.0.0",
  "type": "module",
  "dependencies": {
    "lowdb": "^5.0.5"
  },
  "devDependencies": {
    "@sindresorhus/tsconfig": "^3.0.1",
    "typescript": "^4.8.4"
  }
}

tsconfig.json

{
  "extends": "@sindresorhus/tsconfig",
  "compilerOptions": {
    "outDir": "./lib"
  }
}

src/index.ts

import { Low } from 'lowdb'
import { JSONFile } from 'lowdb/node'
/tmp/lowdb-test is 📦 v1.0.0 via  v19.0.0 
❯ npm x tsc
src/index.ts:1:1 - error TS6133: 'Low' is declared but its value is never read.

1 import { Low } from 'lowdb'
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~

src/index.ts:2:1 - error TS6133: 'JSONFile' is declared but its value is never read.

2 import { JSONFile } from 'lowdb/node'
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Found 2 errors in the same file, starting at: src/index.ts:1
Trystan-SA commented 1 year ago

Still not working. For import {JSONFile} from 'lowdb/lib/node' I get :

X [ERROR] Could not resolve "lowdb/lib/node"
  The path "./lib/node" is not exported by package "lowdb":
  The file "./lib/node.js" is exported at path "./node":
  Import from "lowdb/node" to get the file "node_modules/lowdb/lib/node.js":
 You can mark the path "lowdb/lib/node" as external to exclude it from the bundle, which will
  remove this error.

If I do import {JSONFile} from 'lowdb/node it says the path is not found. Probably because it resolve lowdb package starting from the basepath and not from lowdb/lib folder.

typicode commented 1 year ago

Hmm I don't know why you're still getting this error. You can check https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c#im-having-problems-with-esm-and-typescript

Mapping from lowdb/node to lowdb/lib/node.js is done here btw: https://github.com/typicode/lowdb/blob/main/package.json#L32

tkafka commented 1 year ago

+1 here - typescript project with Next.js.

import { JSONFile } from 'lowdb/node.js': When running next build, this fails with Type error: Cannot find module 'lowdb/node.js' or its corresponding type declarations. in Linting and checking validity of types phase.

import { JSONFile } from 'lowdb/lib/node.js' The Linting and checking validity of types phase completes fine, but then Creating an optimized production build fails with Module not found: Package path ./lib/node.js is not exported from package <project-path>/node_modules/lowdb (see exports field in <project-path>/node_modules/lowdb/package.json)

It seems that the assignment "./node": "./lib/node.js", in package.json is something that one of the tools in pipeline cannot handle.

9a-aaaaaaaa commented 1 year ago

It's totally unusable. If we don't fix this

typicode commented 1 year ago

@tkafka it should be import { JSONFile } from 'lowdb/node' (removed .js)

That said, it seems to be tricky with Next.js for some reasons.

tkafka commented 1 year ago

@typicode Yes, it doesn't work with Next.js - next dev runs fine, but next build fails at type checking.

I have made a simplest possible repo to reproduce this: https://github.com/tkafka/lowdb-node-fail

Meanwhile, I keep using "lowdb": "^3.0.0", which works.

Sec-ant commented 1 year ago

I believe this has something to do with the ts es module resolution strategy. I ran into the issue too, but I switched my project to pure esm config later and everything works. I suggest using the latest typescript version and setting moduleResolution to NodeNext in your tsconfig.json:

{
  "compilerOptions": {
    "moduleResolution": "NodeNext",
  }
}

I ran a quick test using @tkafka 's repo: https://github.com/tkafka/lowdb-node-fail

The default moduleResolution is node, if you run npx tsc --noEmit --traceResolution | grep -A50 -m1 lowdb/node, the output will be something like this:

======== Resolving module 'lowdb/node' from '/home/secant/lowdb-node-fail/services/db.ts'. ========
Explicitly specified module resolution kind: 'NodeJs'.
...
======== Module name 'lowdb/node' was not resolved. ========
...

If you change moduleResolution to nodenext, and run the trace command again, the output will be:

======== Resolving module 'lowdb/node' from '/home/secant/lowdb-node-fail/services/db.ts'. ========
Explicitly specified module resolution kind: 'NodeNext'.
...
File '/home/secant/lowdb-node-fail/node_modules/lowdb/lib/node.d.ts' exist - use it as a name resolution result.
Resolving real path for '/home/secant/lowdb-node-fail/node_modules/lowdb/lib/node.d.ts', result '/home/secant/lowdb-node-fail/node_modules/lowdb/lib/node.d.ts'.
======== Module name 'lowdb/node' was successfully resolved to '/home/secant/lowdb-node-fail/node_modules/lowdb/lib/node.d.ts' with Package ID 'lowdb/lib/node.d.ts@5.0.5'. ========
...

which suggests the module resolution succeeded.

And if someone doesn't add type: "module" in his package.json or doesn't use .mjs file extensions, which means he wants to import an es module in a commonjs module, then he will have to use dynamic import: https://stackoverflow.com/a/70396888/4668057

tkafka commented 1 year ago

@Sec-ant Thank you for the research! I am not sure if the case matters, but the documentation says it should be nodenext.

However, the build still seems to fail:

import { Low } from 'lowdb'
import { JSONFile } from 'lowdb/node'

> ./node_modules/.bin/next build

info  - Linting and checking validity of types ..Failed to compile.

./services/db.ts:5:26
Type error: Cannot find module 'lowdb/node' or its corresponding type declarations.

  3 | 
  4 | import { Low } from 'lowdb'
> 5 | import { JSONFile } from 'lowdb/node'
    |                          ^
  6 | 
  7 | const dbFile = join(homedir(), 'db.json')
  8 | const dbAdapter = new JSONFile(dbFile)

Do you think I should report this to Next.js instead?

Sec-ant commented 1 year ago

I am not sure if the case matters

Both nodenext and NodeNext are fine :)

However, the build still seems to fail:

That's disappointing. My guess is that there are some inconsistent behaviours.

Do you think I should report this to Next.js instead?

There seems to be already several issue and discussions: https://github.com/vercel/next.js/issues/35572 https://github.com/vercel/next.js/discussions/41189

typicode commented 1 year ago

I think it would be worth it reporting to Next.js if there's no duplicate issue already.

In any case, AFAICT, lowdb works with Node + TypeScript with the example provided above and package.json config is correct. So I'm not sure what can be done at lowdb's level to make it work with Next.js.

It's fine to stay on v3 meanwhile. v5 is a maintenance update and fixes issues with front-end tools like Vite.

minimusubi commented 1 year ago

I'm able to reproduce this with just lowdb + TypeScript. See https://github.com/Smiley43210/cannot-find-lowdb for minimum repro.

lowdb@5.05 typescript@4.8.4

Sec-ant commented 1 year ago

@Smiley43210 You should use nodenext for moduleResolution in tsconfig.json, like I said here: https://github.com/typicode/lowdb/issues/554#issuecomment-1293154430

minimusubi commented 1 year ago

Sorry, I got confused and thought that was something specific to Next.js

or2008 commented 1 year ago

have the same issue, unfoutrnly setting moduleResolution to 'nodenext' is casuing other issues in my application :(

image
didac-pf commented 1 year ago

Same problem in NestJS (not NextJS). Changing moduleResolution breaks all other imports.

jsternadel commented 1 year ago

Same problem here. Setting "moduleResolution": "NodeNext" breaks several other imports. Rolled back to v4 and everything is working for now.

vedantroy commented 1 year ago

This breaks with V3 as well.

GrinZero commented 1 year ago

try this solution:

useage

import { JSONFile } from 'lowdb/node'
import type { JSONFile as JSONFileType } from 'lowdb/lib/node'

type DBData = {
  hosts: string[]
}
const adapter = new JSONFile<DBData>(path.join(USER_DATA_PATH, 'db.json')) as JSONFileType<DBData>

.d.ts

declare module 'lowdb/node' {
  export function JSONFile<T>(path: string): void
}
// you can add more
GrinZero commented 1 year ago

try this solution:

useage

import { JSONFile } from 'lowdb/node'
import type { JSONFile as JSONFileType } from 'lowdb/lib/node'

type DBData = {
  hosts: string[]
}
const adapter = new JSONFile<DBData>(path.join(USER_DATA_PATH, 'db.json')) as JSONFileType<DBData>

.d.ts

declare module 'lowdb/node' {
  export function JSONFile<T>(path: string): void
}
// you can add more

I took this plan back, not because I couldn't run. It's because I really don't think it's necessary to insist on using such a bad type system, and the demo can't be opened

bartektartanus commented 1 year ago

Same issue with this import: import {LocalStorage} from 'lowdb/browser';

kumpan-david commented 1 year ago

I think this is related to https://github.com/microsoft/TypeScript/issues/50794

Just as @GrinZero did I got it working but with

declare module 'lowdb/node' {
  export * from 'node_modules/lowdb/lib/node';
}
joycollector commented 1 year ago

For we works like this

// lowdb.d.ts
declare module "lowdb/node" {
  export * from "lowdb/lib/node";
}
import { LowSync } from "lowdb";
import { JSONFileSync } from "lowdb/node";

const db = new LowSync(new JSONFileSync<{ items: { name: string }[] }>("data.json"));
tkafka commented 1 year ago

I can confirm @joycollector's solution works for me as well, thank you!

For the record, I have put lowdb.d.ts file into <project_root>/definitions in my next.js (Next 13) app.

SwapnilSoni1999 commented 1 year ago

Same problem

dtroydev commented 1 year ago

Another option is to patch up lowdb (and leave everything else alone):

  1. npm i -D patch-package (if you don't already have it)
  2. add export * from './node.js' to the end of node_modules/lowdb/lib/index.js and node_modules/lowdb/lib/index.d.ts
  3. npx patch-package lowdb
  4. add "postinstall": "patch-package" to scripts in package.json (if you don't already have this or something to that effect)

The last step ensures that everytime you reinstall modules you will get that patch reapplied.

You will now be importing JSONFile from lowdb, ie. import { Low, JSONFile } from 'lowdb';

Valexr commented 1 year ago
declare module 'lowdb/node' {
  export function JSONFile<T>(path: string): void
}

I think, this declaration mast be in lib d.ts files...🤔

Screenshot 2023-01-30 at 15 06 16 this is strange content in `lowdb/lib/node.d.ts...🤔

tsc will declare types fine🤷🏻‍♂️

Screenshot 2023-01-30 at 15 14 03

true declaration live in lowdb/lib/adapters/JSONFile.d.ts

weoreference commented 1 year ago

Same issue here with plain Typescript. This is a deal breaker for me. Something so basic (even shown in the official tutorial) should work with absolutely no issues. Abstract this problem away. Too much configuration from the get-go

fransyozef commented 1 year ago

yeah I second @weoreference First I thought this package would be very nice to have as mock database on local machine. But just trying to install / fix

import { JSONFile } from 'lowdb/node';

is just breaking everything in my code. to bad :(

typicode commented 1 year ago

Could you please retry with lowdb v6.0.1 which contains @Krak798's fix #569

CKGrafico commented 1 year ago

it works now @typicode :)

wxfred commented 1 year ago

Could you please retry with lowdb v6.0.1 which contains @Krak798's fix #569

Still have some problem in electron

colining commented 1 year ago

not work in electron

nulkode commented 1 year ago

could you please give more information about the error?

colining commented 1 year ago

could you please give more information about the error?

I working in erb project,erb

if I call lowdb in main process of electron ,it will throw error like this require() of es module if I call lowdb in render process react, it will throw other error. I think maybe it's cause some thing diff in wepack.config

demo 【https://github.com/colining/erb-lowdb-test

please see db.ts and

setInstalledGame(["1","2"])

@Krak798 thanks a lot

nulkode commented 1 year ago

@colining

Here you are importing JSONFileSync from the wrong path. Check usage here.

- import { JSONFileSync, LowSync } from "lowdb";
+ import { LowSync } from "lowdb";
+ import { JSONFileSync } from "lowdb/node"; // in electron
wxfred commented 1 year ago

@colining

Here you are importing JSONFileSync from the wrong path. Check usage here.

- import { JSONFileSync, LowSync } from "lowdb";
+ import { LowSync } from "lowdb";
+ import { JSONFileSync } from "lowdb/node"; // in electron
// background.js
import { LowSync } from "lowdb"
import { JSONFileSync } from "lowdb/node"
This dependency was not found:

* lowdb/node in ./src/background.js

To install it, you can run: npm install --save lowdb/node
jakeershareef commented 9 months ago

Error in splitPdfPathHandler: [Error: ENOENT: no such file or directory, rename 'C:\Users\jakee\OneDrive\Desktop\lis-scan-docs\desktop-scan-docs\db.scan.json.tmp' -> 'C:\Users\jakee\OneDrive\Desktop\lis-scan-docs\desktop-scan-docs\db\scan.json'] { errno: -4058, code: 'ENOENT', syscall: 'rename', path: 'C:\Users\jakee\OneDrive\Desktop\lis-scan-docs\desktop-scan-docs\db\.scan.json.tmp', dest: 'C:\Users\jakee\OneDrive\Desktop\lis-scan-docs\desktop-scan-docs\db\scan.json' }

i am getting this error while processing files using electron application, how can i resolve this issue.