search

continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
https://docs.continue.dev/
Apache License 2.0
13.29k stars 923 forks source link

[Jetbrains] Error vectordb::voyage-code-2 index - failed to load native library (JetBrains Plugin) #1406

Open mkarolczyk opened 1 month ago

mkarolczyk commented 1 month ago

Before submitting your bug report

Relevant environment info

- OS: MacOS Ventura
- Continue: 0.0.49
- IDE: PHPStorm

Description

'm encountering an error while updating the vectordb::voyage-code-2 index within a JetBrains plugin. The error message suggests that the native library failed to load.

To reproduce

Run IDE

Log output

2024-06-01T08:26:52] Error updating the vectordb::voyage-code-2 index: Error: vectordb: failed to load native library.
  You may need to run `npm install @lancedb/vectordb-darwin-arm64`.

  If that does not work, please file a bug report at https://github.com/lancedb/lancedb/issues

  Source error: Error: Cannot find module '@lancedb/vectordb-darwin-arm64'
Require stack:
- /snapshot/continue-deploy/binary/out/index.js
1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in 'require' call. 2) If you don't want to compile the package/file into executable and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call using process.cwd() or process.execPath.
yakami129 commented 3 weeks ago

During my troubleshooting, I discovered that running npm run build on the binary and installing @lancedb/vectordb-xxx leads to an overwrite issue.

For instance, there's a requirement to bundle the following dependencies into the plugin. However, in practice, only the last one is actually bundled, resulting in a missing dependency issue:

  "darwin-arm64": "@lancedb/vectordb-darwin-arm64",
  "darwin-x64": "@lancedb/vectordb-darwin-x64",
  "linux-arm64": "@lancedb/vectordb-linux-arm64-gnu",
  "linux-x64": "@lancedb/vectordb-linux-x64-gnu",
  "win32-x64": "@lancedb/vectordb-win32-x64-msvc",
  "win32-arm64": "@lancedb/vectordb-win32-x64-msvc",

I resolved this issue by modifying binary/build.js:

.....

async function installNodeModuleInTempDirAndCopyToCurrent(packageName, toCopy) {
  console.log(`Copying ${packageName} to ${toCopy}`);
  // This is a way to install only one package without npm trying to install all the dependencies
  // Create a temporary directory for installing the package
  const adjustedName = toCopy.replace(/^@/, "").replace("/", "-");
  const tempDir = path.join(
    __dirname,
    "tmp",
    `continue-node_modules-${adjustedName}`,
    packageName,
  );

........

By including the packageName in tempDir, I successfully addressed the dependency overwrite issue.

Hope it helps you.