How to use xenova/transformers in VSCode Extension

[theskcd]

Hey guys! I am trying to use xenova/transformers in CodeStory, we roll a vscode extension as well and I am hitting issues with trying to get the import working, here's every flavor of importing the library which I have tried to date.

const TransformersApi = Function('return import("@xenova/transformers")')();
const { pipeline, env } = await TransformersApi;

const { pipeline, env } = await import('@xenova/transformers')

const TransformersApi = require('@xenova/transformers');
const { pipeline, env } = await TransformersApi;

I think the crux of the issue is the node environment which VSCode uses which does not allow any of these to work, and I keep getting the deaded:

Error [ERR_REQUIRE_ESM]: require() of ES Module /Applications/Aide.app/Contents/Resources/app/extensions/codestory/node_modules/@xenova/transformers/src/transformers.js from /Applications/Aide.app/Contents/Resources/app/extensions/codestory/out/llm/embeddings/sentenceTransformers.js not supported.
Instead change the require of transformers.js in /Applications/Aide.app/Contents/Resources/app/extensions/codestory/out/llm/embeddings/sentenceTransformers.js to a dynamic import() which is available in all CommonJS modules.

after checking the js code which is generated, it ends up including the require word:

__importStar(require('@xenova/transformers'))

when I used the first option which was a function I got a very weird error btw:

[Extension Host] TypeError: A dynamic import callback was not specified.
    at new NodeError (node:internal/errors:399:5)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:39:9)
    at eval (eval at <anonymous> (/Applications/Aide.app/Contents/Resources/app/extensions/codestory/out/llm/embeddings/sentenceTransformers.js:46:41), <anonymous>:3:1)

this is mostly comping from the node version which VSCode uses itself.

Do you guys have any suggestions on what I can do about this? Thanks!

[theskcd]

I did see there is an issue which is tracking the commonjs rollout for the library, and I am n00b at all things typescript so not sure what I am doing almost always XD

[theskcd]

well F https://github.com/microsoft/vscode/issues/130367 this is not supported by the VM being used for running the extension in VSCode.

[xenova]

If I remember correctly, @kungfooman mentioned a possible fix for this a while back... perhaps he can assist here 😇. This might require updating the build process (or just package.json) to let commonjs know how to consume it.

@theskcd In the meantime, could you try importing the web version (https://cdn.jsdelivr.net/npm/@xenova/transformers@2.6.1)? You won't get the best performance since it will run with WASM (onnxruntime-web instead of onnxruntime-node), but should all you to proceed until we find a proper fix.

[kungfooman]

You can use pure ESM, even though you have to start through with CJS (VSCode limitation...). The trick is to use await import:

/**
 * @param {import('vscode').ExtensionContext} context - The context.
 */
async function activate(context) {
  (await import('./register-hello-world.mjs')).activate(context);
  (await import('./register-hello-tensor.mjs')).activate(context);
}
module.exports.activate = activate;

register-hello-world.mjs:

import {add} from '#customImport';
import {vscode} from '#vscode';
/**
 * This method is called when your extension is activated, which
 * happens the very first time the command is executed.
 * @param {import('vscode').ExtensionContext} context - The context.
 */
export function activate(context) {
  // Use the console to output diagnostic information (console.log) and errors (console.error)
  // This line of code will only be executed once when your extension is activated
  console.log('> register extension.helloWorld');
  // The command has been defined in the package.json file
  // Now provide the implementation of the command with registerCommand
  // The commandId parameter must match the command field in package.json
  const disposable = vscode.commands.registerCommand('extension.helloWorld', () => {
    vscode.window.showInformationMessage(`Hello World! add(10, 20) is ${add(10, 20)}`);
  });
  context.subscriptions.push(disposable);
}

register-hello-tensor.mjs:

import {Tensor} from '@xenova/transformers';
import {vscode} from '#vscode';
/**
 * This method is called when your extension is activated, which
 * happens the very first time the command is executed.
 * @param {import('vscode').ExtensionContext} context - The context.
 */
export function activate(context) {
  // Use the console to output diagnostic information (console.log) and errors (console.error)
  // This line of code will only be executed once when your extension is activated
  console.log('> register extension.helloTensor');
  // The command has been defined in the package.json file
  // Now provide the implementation of the command with registerCommand
  // The commandId parameter must match the command field in package.json
  const disposable = vscode.commands.registerCommand('extension.helloTensor', () => {
    const tensor = new Tensor("float32", [1, 2, 3, 4], [2, 2]);
    vscode.window.showInformationMessage('Hello Tensor! ' + tensor.data);
  });
  context.subscriptions.push(disposable);
}

Now you may wonder - what is that import {vscode} from '#vscode';?

You can basically use your package.json like an importmap:

{
  "name": "vscode-extension-transformers",
  "displayName": "vscode-extension-transformers",
  "description": "Transformers.js example for VS Code",
  "version": "1.0.0",
  "publisher": "kungfooman",
  "repository": "https://github.com/kungfooman/vscode-extension-transformers/",
  "scripts": {
    "watch": "echo Begone, watch build step! Embracing true ESM power!",
    "vscode:prepublish": "echo Begone, prepublish build step! Embracing true ESM power!",
    "lint": "eslint \"src/**/*.js\""
  },
  "engines": {
    "vscode": "^1.74.0"
  },
  "categories": [
    "Other"
  ],
  "activationEvents": [],
  "main": "./src/extension.js",
  "contributes": {
    "commands": [
      {
        "command": "extension.helloWorld",
        "title": "Hello World"
      },
      {
        "command": "extension.helloTensor",
        "title": "Hello Tensor"
      }
    ]
  },
  "devDependencies": {
    "@types/node": "^16.18.34",
    "@types/vscode": "^1.73.0",
    "eslint": "^8.26.0"
  },
  "imports": {
    "#customImport": "./src/customImport.mjs",
    "#vscode": "./src/vscode.mjs"
  },
  "dependencies": {
    "@xenova/transformers": "^2.6.1"
  }
}

And ./src/vscode.mjs has to look like this:

import {createRequire} from 'node:module';
const require = createRequire(import.meta.url);
export const vscode = require('vscode');

Tada, problems solved - have fun with ESM. If you use TypeScript, drop it now: it just creates extra headaches through AST transformations etc.

Too much code? I made a complete test repo here:

https://github.com/kungfooman/vscode-extension-transformers

Just clone and open in VSCode and press F5. Typing is done through JSDoc - Pure ESM code, full typing, no TS build step headaches:

[wszgrcy]

You can use pure ESM, even though you have to start through with CJS (VSCode limitation...). The trick is to use await import:

/**
 * @param {import('vscode').ExtensionContext} context - The context.
 */
async function activate(context) {
  (await import('./register-hello-world.mjs')).activate(context);
  (await import('./register-hello-tensor.mjs')).activate(context);
}
module.exports.activate = activate;

register-hello-world.mjs:

import {add} from '#customImport';
import {vscode} from '#vscode';
/**
 * This method is called when your extension is activated, which
 * happens the very first time the command is executed.
 * @param {import('vscode').ExtensionContext} context - The context.
 */
export function activate(context) {
  // Use the console to output diagnostic information (console.log) and errors (console.error)
  // This line of code will only be executed once when your extension is activated
  console.log('> register extension.helloWorld');
  // The command has been defined in the package.json file
  // Now provide the implementation of the command with registerCommand
  // The commandId parameter must match the command field in package.json
  const disposable = vscode.commands.registerCommand('extension.helloWorld', () => {
    vscode.window.showInformationMessage(`Hello World! add(10, 20) is ${add(10, 20)}`);
  });
  context.subscriptions.push(disposable);
}

register-hello-tensor.mjs:

import {Tensor} from '@xenova/transformers';
import {vscode} from '#vscode';
/**
 * This method is called when your extension is activated, which
 * happens the very first time the command is executed.
 * @param {import('vscode').ExtensionContext} context - The context.
 */
export function activate(context) {
  // Use the console to output diagnostic information (console.log) and errors (console.error)
  // This line of code will only be executed once when your extension is activated
  console.log('> register extension.helloTensor');
  // The command has been defined in the package.json file
  // Now provide the implementation of the command with registerCommand
  // The commandId parameter must match the command field in package.json
  const disposable = vscode.commands.registerCommand('extension.helloTensor', () => {
    const tensor = new Tensor("float32", [1, 2, 3, 4], [2, 2]);
    vscode.window.showInformationMessage('Hello Tensor! ' + tensor.data);
  });
  context.subscriptions.push(disposable);
}

Now you may wonder - what is that import {vscode} from '#vscode';?

You can basically use your package.json like an importmap:

{
  "name": "vscode-extension-transformers",
  "displayName": "vscode-extension-transformers",
  "description": "Transformers.js example for VS Code",
  "version": "1.0.0",
  "publisher": "kungfooman",
  "repository": "https://github.com/kungfooman/vscode-extension-transformers/",
  "scripts": {
    "watch": "echo Begone, watch build step! Embracing true ESM power!",
    "vscode:prepublish": "echo Begone, prepublish build step! Embracing true ESM power!",
    "lint": "eslint \"src/**/*.js\""
  },
  "engines": {
    "vscode": "^1.74.0"
  },
  "categories": [
    "Other"
  ],
  "activationEvents": [],
  "main": "./src/extension.js",
  "contributes": {
    "commands": [
      {
        "command": "extension.helloWorld",
        "title": "Hello World"
      },
      {
        "command": "extension.helloTensor",
        "title": "Hello Tensor"
      }
    ]
  },
  "devDependencies": {
    "@types/node": "^16.18.34",
    "@types/vscode": "^1.73.0",
    "eslint": "^8.26.0"
  },
  "imports": {
    "#customImport": "./src/customImport.mjs",
    "#vscode": "./src/vscode.mjs"
  },
  "dependencies": {
    "@xenova/transformers": "^2.6.1"
  }
}

And ./src/vscode.mjs has to look like this:

import {createRequire} from 'node:module';
const require = createRequire(import.meta.url);
export const vscode = require('vscode');

Tada, problems solved - have fun with ESM. If you use TypeScript, drop it now: it just creates extra headaches through AST transformations etc.

Too much code? I made a complete test repo here:

https://github.com/kungfooman/vscode-extension-transformers

Just clone and open in VSCode and press F5. Typing is done through JSDoc - Pure ESM code, full typing, no TS build step headaches:

sorry I build with vsce package and install vsix . run command throw error A dynamic import callback was not specified.

[kungfooman]

@wszgrcy Interesting, thank you, the VSCode ESM loading seems stuck since years because they don't update Electron - it happens to work during extension development, but then fails as package, probably the only-AMD extension logic in VSCode at work.

I feel certain it would still work with some informed debugging work, but I can't do that right now.

[FatumaA]

Hi, Any movement on this?