Closed jeffchuber closed 1 year ago
Answer generated by a 🤖
From your description, it appears that you're encountering an issue with Next.js where it's unable to find the module @visheratin/web-ai
. This is likely due to the package not correctly exporting its main file, which is a common problem when using ES6 imports with CommonJS modules.
To address this, you could modify the package.json
file of the @visheratin/web-ai
package to include an exports
field. This field should specify the main file to be exported when the package is imported. Here's an example of how you could set it up:
{
"name": "@visheratin/web-ai",
"version": "1.0.0",
"main": "index.js",
"exports": {
".": "./index.js"
}
}
In this example, the exports
field is indicating that the main file (index.js
) should be exported when the package is imported.
However, if you don't have the ability to modify the package.json
of the @visheratin/web-ai
package, you could try using a dynamic import()
statement instead of a static require()
statement. This can sometimes bypass issues with ES6 imports and CommonJS modules.
Here's an example of how you could adjust the WebAIEmbeddingFunction
class to use a dynamic import()
statement:
export class WebAIEmbeddingFunction implements IEmbeddingFunction {
// ...
constructor(
modality: "text" | "image",
node: boolean,
proxy?: boolean,
wasmPath?: string,
modelID?: string
) {
if (node) {
this.proxy = proxy ? proxy : false;
import("@visheratin/web-ai-node")
.then((module) => {
webAI = module;
})
.catch((e) => {
console.log(e);
throw new Error(
"Please install the @visheratin/web-ai-node package to use the WebAIEmbeddingFunction, `npm install -S @visheratin/web-ai-node`"
);
});
} else {
this.proxy = proxy ? proxy : true;
import("@visheratin/web-ai")
.then((module) => {
webAI = module;
})
.catch((e) => {
console.log(e);
throw new Error(
"Please install the @visheratin/web-ai package to use the WebAIEmbeddingFunction, `npm install -S @visheratin/web-ai`"
);
});
}
// ...
}
// ...
}
In this revised version of the WebAIEmbeddingFunction
class, the require()
statements have been replaced with import()
statements. These import()
statements return a promise that resolves to the imported module, which is then assigned to the webAI
variable.
Please note that this is just one potential solution to your problem. If you find this solution to be correct and believe it could benefit other users, I encourage you to make a pull request. If this solution doesn't align with your experience, I recommend updating to the latest version of the software.
This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
I got this error with using Next JS using the newer app
router
I rolled back to "chromadb": "1.5.2",
and no longer saw the error.
I got this error with using Next JS using the newer
app
routerI rolled back to
"chromadb": "1.5.2",
and no longer saw the error.
Chiming in here - confirmed that rolling back to 1.5.2
has worked around the error on my end.
@cishiv thanks!! if anyone can help me with a small reproducible example - id love to fix this
@cishiv thanks!! if anyone can help me with a small reproducible example - id love to fix this
Hi @jeffchuber
Here is a repo: https://github.com/e-roy/langchain-chatbot-demo
This is currently using "chromadb": "1.5.2"
If I upgrade to 1.5.5
, it will still work, but there will be Module not found
errors in the terminal
If I go through the process of adding each dependency @visheratin/web-ai-node
, @visheratin/web-ai
and encoding
, then I'll get the error from the original post.
I hope this helps
thank you @e-roy ! asking Vercel for help on this
From a quick look I think the problem is that the CJS build of chromadb
was used (dist/main
here instead of dist/module
):
./node_modules/chromadb/dist/main/embeddings/WebAIEmbeddingFunction.js
And then, a require
is called:
require("@visheratin/web-ai")
So it tries to load the CJS build of @visheratin/web-ai
. However, in that package's exports
field, it has only types
and import
conditions for the .
path defined: https://unpkg.com/browse/@visheratin/web-ai@1.2.3/package.json
"exports": {
".": {
"types": "./index.d.ts",
"import": "./index.js"
},
Which doesn't have a CJS build provided for that path at all. To fix, you have to either use import()
to load the ESM build, or add a "main"
field to its package.json, or add a "require"
condition (https://nodejs.org/api/packages.html#conditional-exports).
@shuding, thank you for the lead! I just opened a PR that addresses the issue with require
/import
statements. I tested the fixed version, and it works in both browser and Node.js.
From a quick look I think the problem is that the CJS build of
chromadb
was used (dist/main
here instead ofdist/module
):./node_modules/chromadb/dist/main/embeddings/WebAIEmbeddingFunction.js
And then, a
require
is called:require("@visheratin/web-ai")
So it tries to load the CJS build of
@visheratin/web-ai
. However, in that package'sexports
field, it has onlytypes
andimport
conditions for the.
path defined: https://unpkg.com/browse/@visheratin/web-ai@1.2.3/package.json"exports": { ".": { "types": "./index.d.ts", "import": "./index.js" },
Which doesn't have a CJS build provided for that path at all. To fix, you have to either use
import()
to load the ESM build, or add a"main"
field to its package.json, or add a"require"
condition (https://nodejs.org/api/packages.html#conditional-exports).
You will still have this error when you use the ESM build. See
- wait compiling /_error (client and server)...
- error ../chroma/clients/js/dist/module/embeddings/WebAIEmbeddingFunction.js:132:0
Module not found: Can't resolve '@visheratin/web-ai'
130 | try {
131 | // @ts-ignore
> 132 | const webAI = await import("@visheratin/web-ai");
133 | if (wasmPath) {
134 | webAI.SessionParams.wasmRoot = wasmPath;
135 | }
https://nextjs.org/docs/messages/module-not-found
Import trace for requested module:
../chroma/clients/js/dist/module/index.js
./src/app/page.js
The problem here is, that the package @visheratin/web-ai
is not installed which is intended because this module is optional and there is a try-catch around that import to catch that error at runtime. However, the Next.js build script tries to locate the module during the compile process and throws this error because it is not installed.
@shuding can you confirm that?
I just checked and can confirm that this issue exists. When working on a fix, I didn't consider this use case.
To fix the issue in version 1.5.6 of the package for Next.js, you can add the following to the Webpack config in next.config.js
:
config.resolve.fallback = {
"@visheratin/web-ai": false
};
This way, the app will get compiled.
If somebody knows how to fix this issue in the WebAIEmbeddingFunction
, I'd greatly appreciate the help.
I just checked and can confirm that this issue exists. When working on a fix, I didn't consider this use case.
To fix the issue in version 1.5.6 of the package for Next.js, you can add the following to the Webpack config in
next.config.js
:config.resolve.fallback = { "@visheratin/web-ai": false };
This way, the app will get compiled.
If somebody knows how to fix this issue in the
WebAIEmbeddingFunction
, I'd greatly appreciate the help.
I already fixed the WebAIEmbeddingFunction
in https://github.com/chroma-core/chroma/pull/956
I was having this same issue on a Vue app I'm working on. I found the following script roughly fixes the import problem so my app will build properly. Not a great solution, but works.
input_file="node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js"
output_file="node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js"
while IFS= read -r line; do
if [[ $line == *'require("@visheratin/'* ]]; then
line="//${line}"
fi
echo "$line"
done < "$input_file" > "$output_file"
we are close to fixing this up thanks to some excellent work by @perzeuss ! stand by :)
well unfortunately neither the PR was merged nor does the custom webpack config work for me. Would be great not to have to build a workaround or rolling back to prev. versions to fix the issue.
This should be fixed in the release 1.5.8
- can folks give that a spin?
@jeffchuber The error is continuously displayed in version 1.5.8 when ingest documents.
Module not found: Can't resolve '@visheratin/web-ai-node/multimodal' in '/Users/xxx/Documents/GitHub/langchain/javascript/langchain-nextjs/node_modules/chromadb/dist/module/embeddings'
Import trace for requested module:
./node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js
./node_modules/chromadb/dist/module/index.js
./node_modules/langchain/dist/vectorstores/chroma.js
./node_modules/langchain/vectorstores/chroma.js
./app/api/retrieval/ingest/route.ts
- ┌ POST /api/chat/retrieval 200 in 21499ms
│
├──── POST http://127.0.0.1:7080/api/v1/collections 200 in 32ms (cache: MISS)
│
├── 1 level ── POST http://127.0.0.1:11434/api/generate 200 in 3215ms (cache: MISS)
│
└── 1 level ── POST http://127.0.0.1:7080/api/v1/collections/1038.. 200 in 66ms (cache: MISS)
@jeffchuber The error is continuously displayed in version 1.5.8 when ingest documents.
Module not found: Can't resolve '@visheratin/web-ai-node/multimodal' in '/Users/xxx/Documents/GitHub/langchain/javascript/langchain-nextjs/node_modules/chromadb/dist/module/embeddings' Import trace for requested module: ./node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js ./node_modules/chromadb/dist/module/index.js ./node_modules/langchain/dist/vectorstores/chroma.js ./node_modules/langchain/vectorstores/chroma.js ./app/api/retrieval/ingest/route.ts - ┌ POST /api/chat/retrieval 200 in 21499ms │ ├──── POST http://127.0.0.1:7080/api/v1/collections 200 in 32ms (cache: MISS) │ ├── 1 level ── POST http://127.0.0.1:11434/api/generate 200 in 3215ms (cache: MISS) │ └── 1 level ── POST http://127.0.0.1:7080/api/v1/collections/1038.. 200 in 66ms (cache: MISS)
@jonsoku2 Are you trying to use the webai embedding function or only the chroma client?
This seems to be working fine now with Next JS, using:
"chromadb": "^1.5.9",
"langchain": "^0.0.150",
"next": "^13.4.19",
Here is a working example:
https://github.com/e-roy/langchain-chatbot-demo/tree/main
For this error:
Module not found: Can't resolve '@visheratin/web-ai-node/multimodal' in '/Users/xxx/Documents/GitHub/langchain/javascript/langchain-nextjs/node_modules/chromadb/dist/module/embeddings'
Import trace for requested module:
./node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js
./node_modules/chromadb/dist/module/index.js
./node_modules/langchain/dist/vectorstores/chroma.js
./node_modules/langchain/vectorstores/chroma.js
./app/api/retrieval/ingest/route.ts
You can adjust your next.config.js
file and ignore it by adding
webpack: (config, {}) => {
config.resolve.fallback = {
"@visheratin/web-ai-node": false,
};
return config;
},
This seems to be working fine now with Next JS, using:
"chromadb": "^1.5.9",
"langchain": "^0.0.150",
"next": "^13.4.19",
Here is a working example:
https://github.com/e-roy/langchain-chatbot-demo/tree/main
For this error:
Module not found: Can't resolve '@visheratin/web-ai-node/multimodal' in '/Users/xxx/Documents/GitHub/langchain/javascript/langchain-nextjs/node_modules/chromadb/dist/module/embeddings' Import trace for requested module: ./node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js ./node_modules/chromadb/dist/module/index.js ./node_modules/langchain/dist/vectorstores/chroma.js ./node_modules/langchain/vectorstores/chroma.js ./app/api/retrieval/ingest/route.ts
You can adjust your
next.config.js
file and ignore it by addingwebpack: (config, {}) => { config.resolve.fallback = { "@visheratin/web-ai-node": false, }; return config; },
interesting! Will try out. Thx for notifying!
This is fixed with 1.6.1
From discord: https://discord.com/channels/1073293645303795742/1130258136905814066/1130258136905814066