hmenyus / node-calls-python

Call Python from NodeJS directly in-process without spawning processes
MIT License
252 stars 26 forks source link

python import error `multiprocessing.resource_tracker` #91

Open taoche opened 4 weeks ago

taoche commented 4 weeks ago

When running the test program, I encountered the following error message. It seems to be related to the use of multiprocessing in the sentence_transformers module. I have tried several ways provided by nodecallspython to call Python code, but still encountered this error. I also tried switching between different versions of Node.js and Python, but it doesn't seem to help.

node js version:  20.11.1 / 18.17.1
python version:  3.11.9 / 3.12.4 
Error: Cannot find module '`test-python-project/from multiprocessing.resource_tracker import main;main(31)'
    at Module._resolveFilename (node:internal/modules/cjs/loader:1144:15)
    at node:internal/main/check_syntax:33:20 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

test.py

from sentence_transformers import SentenceTransformer

def sentence(sentence):
    sentences = [sentence]

    model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
    embeddings = model.encode(sentences)
    return embeddings

main.js

import nodecallspython from "node-calls-python";

  // Call python script
  const py = nodecallspython.interpreter;
  py.import("./test.py", false).then(async function (pymodule) {
    const result = await py.call(
      pymodule,
      sentence,
      "sentence sentence sentence"
    );
    console.log(result);
  });
taoche commented 4 weeks ago

update: Using node's child_process const { spawn } = require("child_process") will not result in a multiprocessing error.