Crash when using `import-in-the-middle` with `@langchain/core`: `ReferenceError: RunTree is not defined`

sabrenner commented 2 weeks ago

Expected Behavior

Running node --loader ./hook.mjs index.mjs for

// index.mjs
import { BaseChatModel } from "@langchain/core/language_models/chat_models";

// hook.mjs
export * from 'import-in-the-middle/hook.mjs'

should not error.

Actual Behavior

When running node --loader ./hook.mjs index.mjs, the following error is produced:

file:///Users/sam.brenner/Development/ml-obs/langchain-nodejs/node_modules/langsmith/dist/index.js?iitm=true:6
const _ = Object.assign(
                 ^

ReferenceError: RunTree is not defined
    at Function.assign (<anonymous>)
    at file:///Users/sam.brenner/Development/ml-obs/langchain-nodejs/node_modules/langsmith/dist/index.js?iitm=true:6:18
    at ModuleJob.run (node:internal/modules/esm/module_job:262:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:483:26)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:117:5)

Steps to Reproduce the Problem

Create a new directory, and install the following dependencies:

mkdir langchain-iitm-issue
cd langchain-iitm-issue
npm init -y
npm install @langchain/core import-in-the-middle

Create an index.mjs file, with the following contents

// index.mjs
import { BaseChatModel } from "@langchain/core/language_models/chat_models";

Create a hook.mjs file, with the following contents

// hook.mjs
export * from 'import-in-the-middle/hook.mjs';

Run node --loader ./hook.mjs index.mjs in the current directory.

Specifications

Running npx envinfo:

System:
    OS: macOS 14.7
    CPU: (10) arm64 Apple M1 Max
    Memory: 593.48 MB / 64.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 22.9.0 - /opt/homebrew/bin/node
    Yarn: 1.22.22 - /opt/homebrew/bin/yarn
    npm: 10.8.3 - /opt/homebrew/bin/npm

Investigation

Context: I am working on writing some instrumentation code for LangChain, and wanted to verify it works with the import-in-the-middle hooks for ESM.

I tried to do a bit of debugging to see what exactly was going on. Pointing the hook instead to my local version of import-in-the-middle, I tried to log the namespace we're re-exporting from. Doing that, I can see the namespace that's throwing the error is:

[Module: null prototype] {
  Client: [class Client],
  RunTree: <uninitialized>,
  __version__: '0.2.3',
  overrideFetchImplementation: [Function: overrideFetchImplementation]
}

Where it seems the problematic RunTree property is <uninitialized>. I'm not entirely sure how this happens, or if it's a product of how import-in-the-middle is interacting with the transpiled code. The original file is defined here.

Naive/noob question: would it be possible to just disable ESM parsing/loader hooks for certain files? This problematic file, from langsmith, is a sub-dependency of langchain, and is not a file I would want to instrument.

I've tried changing my hook.mjs to be:

import * as module from 'module'

module.register('import-in-the-middle/hook.mjs', import.meta.url, {
  data: { exclude: ['langsmith'] }
})

Although this does not seem to be helpful, although it's likely I'm doing it wrong 😅 Also, my understanding is that will only work for newer versions of Node, and this integration I'm writing should be compatible back through Node 16. Any insights here are appreciated, happy to follow up with any additional information!

sabrenner commented 2 weeks ago

I realized that the workaround to disable ESM alltogether should be something like:

// register.mjs
import * as module from 'module'

module.register('import-in-the-middle/hook.mjs', import.meta.url, {
  data: { exclude: [/langsmith/, /openai/] } // excluding openai as well, as there is another IITM issue with that
})

There still seems to be some problem with instrumenting LangChain properly, but I believe that is on me to figure out, otherwise I will follow up.

I am wondering in general if there are ways to accomplish this for the --loader method, where we cannot leverage the kind of functionality in register.mjs above, for "older" (back through 16) versions of Node. Additionally, any insight to the original issue is also appreciated, thanks! 😄

timfish commented 2 weeks ago

I suspect the issue you are seeing is a duplicate of one of the existing known issues. So #38 or similar to #141.

import-in-the-middle wraps modules in additional modules. It tries to create matching exports so the wrapping appears invisible but it is unable to fully replicate the behaviour of a module in a couple of cases.

nodejs / import-in-the-middle