[Bug]: Module parse failed: Unexpected character '�'

[dsws]

What happened?

When I try to invoke this node file with import, it complains "To load an ES module, set "type": "module" in the package.json or use the .mjs extension."

import { ChromaClient } from "chromadb"; const chroma = new ChromaClient({ path: "http://localhost:8000" });

When i add the line to my package.json file all works fine.

"type": "module"

However, when I try to load the same in my NextJS app... it's giving me the following error

./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node Module parse failed: Unexpected character '�' (1:0) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders (Source code omitted for this binary file)

Import trace for requested module: ./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node ./node_modules/onnxruntime-node/bin/napi-v3/ sync ^.\/.\/.\/onnxruntime_binding.node$ ./node_modules/onnxruntime-node/dist/binding.js ./node_modules/onnxruntime-node/dist/backend.js ./node_modules/onnxruntime-node/dist/index.js ./node_modules/@visheratin/web-ai-node/node/session.js ./node_modules/@visheratin/web-ai-node/node/index.js ./node_modules/@visheratin/web-ai-node/index.js ./node_modules/chromadb/dist/module/embeddings/WebAIEmbeddingFunction.js ./node_modules/chromadb/dist/module/index.js ./app/api/chat/retrieval/route.ts

Versions

"chromadb": "1.5.11",

Relevant log output

No response

[jeffchuber]

@dsws i tried 1.5.11 and nextjs this morning and it worked for me - is it possible to post a minimal repro? that would help me fix this. thanks!

[dsws]

Hey Jeff, really appreciate your help on this... I pushed the updated repo here

This is the file I modified https://github.com/dsws/langchain-nextjs-template/blob/main/app/api/chat/retrieval/route.ts

If you want to see original its in the project langchain-nextjs-template

If you run yarn && yarn dev then submit the form here http://localhost:3000/retrieval

[tazarov]

@jeffchuber @dsws, I think this issue has been flying around for a week or so where "type": "module" needs to be added to Chroma package.json

[tazarov]

@dsws, can you make the above change in your local copy of the project dependency and retry?

[dsws]

@tazarov added this line to node_modules/chromadb/package.json

But still the same issue....

@jeffchuber have u had a chance to run my project?

[dsws]

Actually i removed my webpack config and it started to not throw an error anymore, but the process takes a very long time.

My embedding are tiny from 3 documents

        const vectorStore = await Chroma.fromDocuments(documents, embeddings, {
            collectionName: "a-test-collection",
            url: "http://127.0.0.1:8000", // Optional, will default to this value
            collectionMetadata: {
                "hnsw:space": "cosine",
            },
        });

The process either takes a very long time, or just hanging...

Thoughts?

[dsws]

After installing encoding it started to give my original error again... so this is still and issue

[l4b4r4b4b4]

Error came back after I upadted to latest chroma. Used the next.js primer that @tazarov gave me, but still same error: ./node_modules/.pnpm/onnxruntime-node@1.14.0/node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node Module parse failed: Unexpected character '�' (1:0) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders (Source code omitted for this binary file)

Is onnxruntime something I need to install on the OS?

[thomas-servais]

Hi,

same issue here (chromadb: ⨯ ./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node Module parse failed: Unexpected character '�' (1:0) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders (Source code omitted for this binary file)

Import trace for requested module: ./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node ./node_modules/onnxruntime-node/bin/napi-v3/ sync ^.\/.\/.\/onnxruntime_binding.node$ ./node_modules/onnxruntime-node/dist/binding.js ./node_modules/onnxruntime-node/dist/backend.js ./node_modules/onnxruntime-node/dist/index.js ./node_modules/chromadb-default-embed/src/backends/onnx.js ./node_modules/chromadb-default-embed/src/env.js ./node_modules/chromadb-default-embed/src/transformers.js ./node_modules/chromadb/dist/chromadb.mjs ./node_modules/@langchain/community/dist/vectorstores/chroma.js ./node_modules/@langchain/community/vectorstores/chroma.js ./node_modules/langchain/dist/vectorstores/chroma.js ./node_modules/langchain/vectorstores/chroma.js

"chromadb": "^1.8.1", "chromadb-default-embed": "^2.13.2",

[nneven]

Hi, running into same issue, any solutions?

[tazarov]

@nneven, have a look at this discord convo - https://discord.com/channels/1073293645303795742/1215704892010663936

If you have a nextjs app the error indicates that your browser doesn't know how to read the onnxruntime binary which is necessary for the default embedding function in Chroma JS client.

[irian-codes]

Okay so I solved the issue on my end thanks to this StackOverflow answer. From what I understand Chroma has a dependency (onnxruntime-node) that uses .node files and webpack needs an external tool to load those. However, here they fixed it by just flat out ignoring them (but since the node-loader solution works I'm not trying this other one).

From the solution I had to omit this code or it wouldn't compile.

if (!isServer) {
      // Fixes npm packages that depend on `fs` module
      config.resolve.fallback.fs = false;
    }

Also I had to install the package @xenova/transformers.

And now I could embed stuff into the database! 🥳

[jferrettiboke]

I'm experiencing the same issue.

"chromadb": "^1.8.1",
"chromadb-default-embed": "^2.13.2",
"next": "14.2.5",

Node.js v20.12.1.

[jferrettiboke]

I've placed this within my next.config.mjs.

  webpack: (config, { isServer }) => {
    if (!isServer) {
      // Fixes npm packages that depend on `fs` module
      config.resolve.fallback.fs = false;
    }

    config.module.rules.push({
      test: /\.node$/,
      use: "node-loader",
    });

    return config;
  },

However, I'm getting the following error even though I've installed chromadb-default-embed.

Error: Please install chromadb-default-embed as a dependency with, e.g. `yarn add chromadb-default-embed`
    at _DefaultEmbeddingFunction.import (webpack-internal:///(rsc)/./node_modules/.pnpm/chromadb@1.8.1_encoding@0.1.13_openai@4.53.2_encoding@0.1.13_/node_modules/chromadb/dist/chromadb.mjs:2450:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async _DefaultEmbeddingFunction.loadClient (webpack-internal:///(rsc)/./node_modules/.pnpm/chromadb@1.8.1_encoding@0.1.13_openai@4.53.2_encoding@0.1.13_/node_modules/chromadb/dist/chromadb.mjs:2434:26)
    at async _DefaultEmbeddingFunction.generate (webpack-internal:///(rsc)/./node_modules/.pnpm/chromadb@1.8.1_encoding@0.1.13_openai@4.53.2_encoding@0.1.13_/node_modules/chromadb/dist/chromadb.mjs:2408:5)
    at async Collection.validate (webpack-internal:///(rsc)/./node_modules/.pnpm/chromadb@1.8.1_encoding@0.1.13_openai@4.53.2_encoding@0.1.13_/node_modules/chromadb/dist/chromadb.mjs:1775:22)
    at async Collection.add (webpack-internal:///(rsc)/./node_modules/.pnpm/chromadb@1.8.1_encoding@0.1.13_openai@4.53.2_encoding@0.1.13_/node_modules/chromadb/dist/chromadb.mjs:1844:73)

[l4b4r4b4b4]

Hi,

same issue here (chromadb: ⨯ ./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node Module parse failed: Unexpected character '�' (1:0) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders (Source code omitted for this binary file)

Import trace for requested module: ./node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node ./nodemodules/onnxruntime-node/bin/napi-v3/ sync ^././._/onnxruntime_binding.node$ ./node_modules/onnxruntime-node/dist/binding.js ./node_modules/onnxruntime-node/dist/backend.js ./node_modules/onnxruntime-node/dist/index.js ./node_modules/chromadb-default-embed/src/backends/onnx.js ./node_modules/chromadb-default-embed/src/env.js ./node_modules/chromadb-default-embed/src/transformers.js ./node_modules/chromadb/dist/chromadb.mjs ./node_modules/@langchain/community/dist/vectorstores/chroma.js ./node_modules/@langchain/community/vectorstores/chroma.js ./node_modules/langchain/dist/vectorstores/chroma.js ./node_modules/langchain/vectorstores/chroma.js

"chromadb": "^1.8.1", "chromadb-default-embed": "^2.13.2",

I again encountered this issue when trying to use xenova/transformers.

Solution was: Use npm and NOT pnpm, yarn or some other package manager.

[Ataxia123]

Ive traid most of these solutions to no avail. Putting it as module would break everything on my end. Let me know if you manage it guys!