TypeError: Cannot read properties of undefined (reading 'create')

vnikolaew commented 5 months ago

System Info

"@xenova/transformers": "^2.17.1", "onnxruntime-node": "^1.17.0", Windows 11 Chrome ❯ npm --version 10.5.2 ❯ node --version v22.0.0

Next.js v14.0.4

Environment/Platform

[X] Website/web-app
[ ] Browser extension
[X] Server-side (e.g., Node.js, Deno, Bun)
[ ] Desktop app (e.g., Electron)
[ ] Other (e.g., VSCode extension)

Description

I am trying to use the pipeline API in my Nextjs app ( more specifically in a NextJS API Route). But I am getting the below error. I am not sure what I am doing wrong.

TypeError: Cannot read properties of undefined (reading 'create')
    at constructSession (webpack-internal:///(rsc)/../../node_modules/@xenova/transformers/src/models.js:440:39)
    at async Promise.all (index 1)
    at async BertModel.from_pretrained (webpack-internal:///(rsc)/../../node_modules/@xenova/transformers/src/models.js:1001:20)
    at async AutoModel.from_pretrained (webpack-internal:///(rsc)/../../node_modules/@xenova/transformers/src/models.js:4993:20)
    at async Promise.all (index 1)
    at async loadItems (webpack-internal:///(rsc)/../../node_modules/@xenova/transformers/src/pipelines.js:2837:5)
    at async pipeline (webpack-internal:///(rsc)/../../node_modules/@xenova/transformers/src/pipelines.js:2789:21)
    at async SentenceSimilarity.run (webpack-internal:///(rsc)/../worker/src/lib/tasks/SentenceSimilarity.ts:21:30)
    at async ClassifyImagesWorker.processCore (webpack-internal:///(rsc)/../worker/src/lib/ClassifyImagesWorker.ts:83:20)
    at async time (webpack-internal:///(rsc)/./app/api/utils.ts:10:20)
    at async V1InngestExecution.eval [as userFnToRun] (webpack-internal:///(rsc)/./app/api/inngest/functions.ts:25:5

This is my next.config.mjs:

// eslint-disable-next-line @typescript-eslint/no-var-requires
const { composePlugins, withNx } = require("@nx/next");

/**
 * @type {import("@nx/next/plugins/with-nx").WithNxOptions}
 **/
const nextConfig = {
   output: `standalone`,
   nx: {
      // Set this to true if you would like to use SVGR
      // See: https://github.com/gregberge/svgr
      svgr: false,

   },
   typescript: {
      ignoreBuildErrors: true,
   },
   sentry: {
      disableServerWebpackPlugin: true,
      disableClientWebpackPlugin: true,
   },
   experimental: {
      serverComponentsExternalPackages: ["sharp", "onnxruntime-node"],
   },
   webpack: (config) => {
      // See https://webpack.js.org/configuration/resolve/#resolvealias
      config.externals = [...config.externals, "hnswlib-node"];

      config.resolve.alias = {
         ...config.resolve.alias,
         "sharp$": false,
         "onnxruntime-node$": false,
      };
      return config;
   },
   images: {
      remotePatterns: [
         {
            hostname: "lh3.googleusercontent.com",
            protocol: `https`,
         },
         {
            hostname: "randomuser.me",
            protocol: `https`,
         },
         {
            hostname: "cdn.{APP_NAME}.com",
            protocol: `https`,
         },
         {
            hostname: "{APP_NAME}.com",
            protocol: `https`,
         },
         {
            hostname: "files.stripe.com",
            protocol: `https`,
         },
         {
            hostname: "live.staticflickr.com",
            protocol: `https`,
         },
         {
            hostname: "staticflickr.com",
            protocol: `https`,
         },
         {
            hostname: "cdn.pixabay.com",
            protocol: `https`,
         },
         {
            hostname: "www.dropbox.com",
            protocol: `https`,
         },
      ],
   },
};

const plugins = [
   // Add more Next.js plugins to this list if needed.
   withNx,
];

module.exports = composePlugins(...plugins)(nextConfig);

I am calling pipeline server-side:

this.extractor = await pipeline(
            "feature-extraction",
             "Snowflake/snowflake-arctic-embed-s",
            {
               quantized: false, // Comment out this line to use the quantized version
            });

Any help/recommendation would be highly appreciated!

Reproduction

create next.js app
install @xenova/transformer npm package
call pipeline API in any API route

kallebysantos commented 5 months ago

I'm getting similar error TypeError: Cannot read properties of undefined (reading 'constructor')

EDIT: was a mistake in the Supabase code that I'd follow. I already fix it

xenova commented 5 months ago

Can you try remove these lines (since you already have set them as external):

      config.resolve.alias = {
         ...config.resolve.alias,
         "sharp$": false,
         "onnxruntime-node$": false,
      };

I think this is the problem, which ignores these packages.

vnikolaew commented 5 months ago

Now I get the following error when commenting out these lines:

 ⨯ ../../node_modules/@xenova/transformers/node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
(Source code omitted for this binary file)

Import trace for requested module:
../../node_modules/@xenova/transformers/node_modules/onnxruntime-node/bin/napi-v3/darwin/arm64/onnxruntime_binding.node
../../node_modules/@xenova/transformers/node_modules/onnxruntime-node/bin/napi-v3/ sync ^\.\/.*\/.*\/onnxruntime_binding\.node$
../../node_modules/@xenova/transformers/node_modules/onnxruntime-node/dist/binding.js
../../node_modules/@xenova/transformers/node_modules/onnxruntime-node/dist/backend.js
../../node_modules/@xenova/transformers/node_modules/onnxruntime-node/dist/index.js
../../node_modules/@xenova/transformers/src/backends/onnx.js
../../node_modules/@xenova/transformers/src/env.js
../../node_modules/@xenova/transformers/src/transformers.js

antem66 commented 5 months ago

Any updates on this? Im facing exactly the same issue.

xenova commented 5 months ago

It looks like Next.js v14 has removed experimental.serverComponentsExternalPackages. It is now https://nextjs.org/docs/app/api-reference/next-config-js/serverExternalPackages. Could you try that?

xenova commented 5 months ago

Within the past week, it's back to https://nextjs.org/docs/app/api-reference/next-config-js/serverComponentsExternalPackages.

I've updated the example apps to Next.js v14, which you can follow to ensure you don't run into this issue.

btahir commented 4 months ago

Unfortunately this is still happening. Any updates?

I am trying to generate embeddings on a server component.

import { pipeline } from '@xenova/transformers';

// Use the Singleton pattern to enable lazy construction of the pipeline.
// NOTE: We wrap the class in a function to prevent code duplication (see below).
const P = () => class PipelineSingleton {
    static task = 'feature-extraction';
    static model = 'Supabase/gte-small';
    static instance = null;

    static async getInstance(progress_callback = null) {
        if (this.instance === null) {
            this.instance = await pipeline(this.task, this.model, { progress_callback });
        }
        return this.instance;
    }
}

let PipelineSingleton;
if (process.env.NODE_ENV !== 'production') {
    // When running in development mode, attach the pipeline to the
    // global object so that it's preserved between hot reloads.
    // For more information, see https://vercel.com/guides/nextjs-prisma-postgres
    if (!global.PipelineSingleton) {
        global.PipelineSingleton = P();
    }
    PipelineSingleton = global.PipelineSingleton;
} else {
    PipelineSingleton = P();
}

export default PipelineSingleton;

And I call it like this

    const embeddingPipeline = await PipelineSingleton.getInstance()
    const output = await embeddingPipeline(text, {
      pooling: 'mean',
      normalize: true,
    })
    const embedding = Array.from(output.data)

I have to include webpack code as well even though yours just had the experimental flag otherwise I get the same error as the person above.

/** @type {import('next').NextConfig} */
const nextConfig = {
  experimental: {
    serverComponentsExternalPackages: ['sharp', 'onnxruntime-node'],
  },

  webpack: (config) => {
    // Ignore node-specific modules when bundling for the browser
    // See https://webpack.js.org/configuration/resolve/#resolvealias
    config.resolve.alias = {
      ...config.resolve.alias,
      sharp$: false,
      'onnxruntime-node$': false,
    }
    return config
  },
}

module.exports = nextConfig

Appreciate any assistance. Thanks!

xenova commented 4 months ago

If you are running the models in-browser, you should only require the webpack: ... settings. If you are running the models server-side, you should only require the serverComponentsExternalPackages: ....

Unfortunately, this isn't a Transformers.js issue and is more related to bundling. If this issue persists, it might be best to ask the question to the Next.js community 😇

btahir commented 4 months ago

That's fair - but it does break all Next.js starters for Transformers.js unfortunately. I'll try and see if someone on that side has addressed it.

xenova commented 4 months ago

Please refer to our Next.js starter examples:

Server-side: https://github.com/xenova/transformers.js/tree/main/examples/next-server
Client-side: https://github.com/xenova/transformers.js/tree/main/examples/next-client

This may assist you :)

btahir commented 4 months ago

This is the error I get starting up the next server example. As mentioned before taking out webpack causes this error which means serverComponentsExternalPackages does not do anything.

If you add the webpack lines back in next.config you end up with the same error: TypeError: Cannot read properties of undefined (reading 'create').

I know its frustrating to maintain parity with the constant changes in Next but things are broken and this is one of the more popular ways users will access transformers.js.

xenova commented 4 months ago

Can you try with npm and not pnpm? I tested the other day and it seems to be working correctly 👀

btahir commented 4 months ago

Woa that actually worked. Very bizarre.

Thanks for debugging!

rafheros commented 3 months ago

Solved the issue above about the char but this error appeared instead, I'm using HuggingfaceEmbedding class from Langchain

TypeError: text.replace is not a function at HuggingFaceTransformersEmbeddings.embedQuery (webpack-internal:///(rsc)/./node_modules/@langchain/community/dist/embeddings/hf_transformers.js:86:39) at ElasticVectorSearch.similaritySearch (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/vectorstores.js:112:90) at VectorStoreRetriever._getRelevantDocuments (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/vectorstores.js:76:33) at VectorStoreRetriever.getRelevantDocuments (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/retrievers/index.js:76:40) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async VectorStoreRetriever._streamIterator (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:188:9) at async VectorStoreRetriever.transform (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:419:9) at async wrapInputForTracing (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:292:30) at async pipeGeneratorWithSetup (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/utils/stream.js:248:19) at async RunnableLambda._transformStreamWithConfig (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:313:26) at async RunnableSequence._streamIterator (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:1214:30) at async RunnableSequence.transform (webpack-internal:///(rsc)/./node_modules/@langchain/core/dist/runnables/base.js:419:9)

gaspar09 commented 3 months ago

For the original error, the issue is because InferenceSession is undefined (causing the undefined.create TypeError) on the imported onnxruntime-node module on https://github.com/xenova/transformers.js/blob/main/src/backends/onnx.js#L34

A workaround I found was to install the onnxruntime-node@1.14.0 dependency directly

rosman21 commented 2 months ago

is there a workaround with getting this working with pnpm?

YupengLing77 commented 3 weeks ago

I installed the onnxruntime-node@1.14.0 dependency directly and remove the

config.resolve.alias = {
         ...config.resolve.alias,
         "sharp$": false,
         "onnxruntime-node$": false,
      };

and the same problem

Module parse failed: Unexpected character '�' (1:0)

happened

MohamedIdrissi3006 commented 1 week ago

I'm getting similar error TypeError: Cannot read properties of undefined (reading 'constructor')

EDIT: was a mistake in the Supabase code that I'd follow. I already fix it

how did you fixed it please

huggingface / transformers.js