dims undefined when converting own model to ONNX

[khromov]

System Info

└─┬ llamaindex@0.1.10
  └── @xenova/transformers@2.15.0

Environment/Platform

• [ ] Website/web-app
• [ ] Browser extension
• [X] Server-side (e.g., Node.js, Deno, Bun)
• [ ] Desktop app (e.g., Electron)
• [ ] Other (e.g., VSCode extension)

Description

After trying to use my own quantized model, I'm getting this error:

TypeError: Cannot read properties of undefined (reading 'dims')
    at mean_pooling (file:///Users/x/Documents/GitHub/llm-newsflow/node_modules/@xenova/transformers/src/utils/tensor.js:729:36)
    at Function._call (file:///Users/x/Documents/GitHub/llm-newsflow/node_modules/@xenova/transformers/src/pipelines.js:1194:22)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Reproduction

First I'm converting an existing model to ONNX

python -m convert --model_id sentence-transformers/paraphrase-multilingual-mpnet-base-v2

Then I'm using the HuggingFace embedding from Llamaindex (although the pipeline is set up by transformers directly):

import {
 HuggingFaceEmbedding
} from "llamaindex";

const MODEL = "sentence-transformers/paraphrase-multilingual-mpnet-base-v2";

//...
  const embedModel = new HuggingFaceEmbedding({
    modelType:  MODEL,
    getExtractor: async () => {
        const { pipeline } = await import("@xenova/transformers");
        const p = await pipeline("feature-extraction", MODEL, {
            quantized: false,
            local_files_only: true,
            cache_dir: './models',
        });

        return p;
    }
  });

The HuggingFaceEmbedding class is just a thin wrapper around transformers.

The above code works fine with existing models on HF.

[xenova]

Hi there 👋 Yes, I noticed this recently when converting recent sentence-transformers models with Optimum, and it should be fixed by this PR. In the meantime, you can use the already-converted version of sentence-transformers/paraphrase-multilingual-mpnet-base-v2, which is Xenova/paraphrase-multilingual-mpnet-base-v2 (link).