Using document loaders from an Electron Application.

[jwhenry28]

As per the LangChain JS docs, most of the document loaders require NodeJS to run properly.

Unfortunately, this seems to prevent their use in Electron application, which (while desktop applications) technically run via Chromium. For example, if I include the following code in an Electron application:

let infile = this.baseDir + "/TestFile.md";
this.loader = new TextLoader(infile);
const docs = await this.loader.load();

I get the following error:

Error: Failed to load fs/promises. TextLoader available only on environment 'node'. It appears you are running environment 'browser'. See https://<link to docs> for alternatives.

Looking in the source code, I can see this error triggered here: https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/document_loaders/fs/text.ts#L88

try {
  const { readFile } = await import("node:fs/promises");
  return { readFile };
} catch (e) {
  console.error(e);
  throw new Error(
    `Failed to load fs/promises. TextLoader available only on environment 'node'. It appears you are running environment '${getEnv()}'. See https://<link to docs> for alternatives.`
  );
}

Is there a workaround that we can use to call TextLoader or similar document loaders from an Electron application?

And for reference:

$ npm list
...
├── langchain@0.0.211
...

[dosubot[bot]]

🤖

Hi @jwhenry28! 👋

I'm Dosu, an AI here to lend a hand with your LangChain JS questions and challenges. While we wait for a human maintainer, I'm available to help troubleshoot bugs, provide answers, and even guide you on how to become a contributor. Let's dive into the world of code together! 🚀

It seems like you're trying to use the TextLoader in an Electron environment, which is a bit different from a standard Node.js environment. Electron applications can use Node.js modules directly, as they are built on top of Node.js and Chromium.

Here's how you can import and use the TextLoader in an Electron application:

const { TextLoader } = require('langchain/document_loaders/fs/text');

let loader = new TextLoader();
let document = loader.load('path/to/your/textfile.txt');

Please replace 'path/to/your/textfile.txt' with the actual path to your text file. This code will load the text file into the document variable.

The TextLoader class in the LangChain JS framework is used to load text data from a file. It takes a file path as an argument during instantiation. The load method is then used to read the file and load the content. The loaded content is returned as an array of documents, where each document is an object with two properties: pageContent and metadata. The pageContent property contains the text content of the file, and the metadata property contains an object with a source key that holds the path of the source file.

You can find more details about the TextLoader class in the langchain/src/document_loaders/tests/text.test.ts file.

Please note that you're using an older version of LangChain JS (0.0.211). The latest version is 0.0.212. It might be a good idea to update to the latest version to ensure you have the latest features and bug fixes. You can do this by running npm install langchain@latest in your Electron application's directory.

Let me know if this helps or if you have any other questions.

Sources

• langchain/src/load/import_constants.ts
• langchain/src/document_loaders/tests/text.test.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jwhenry28]

This solution above did not work.

After looking closer at the LangChain source code for TextLoader, I found a way to avoid the call to readFile:

if (typeof this.filePathOrBlob === "string") {
  const { readFile } = await TextLoader.imports();
  text = await readFile(this.filePathOrBlob, "utf8");
  metadata = { source: this.filePathOrBlob };
} else {
  text = await this.filePathOrBlob.text();
  metadata = { source: "blob", blobType: this.filePathOrBlob.type };
}

So if we pass in a Blob to the loader instead of a file name, it seems to work:

let infile = this.baseDir + "/TestFile.md";
const fileBlob = await fileToBlob(infile); // custom helper function to read a file as a Blob from Electron app
const fileText = await fileBlob.text();
this.loader = new TextLoader(fileBlob);
const docs = await this.loader.load();

Please let me know if there is a better solution.

[dosubot[bot]]

Hi, @jwhenry28,

I'm helping the langchainjs team manage their backlog and am marking this issue as stale. It seems that you found a workaround by passing a Blob to the TextLoader instead of a file name, which resolved the issue you were encountering in an Electron application. You mentioned seeking feedback on whether there is a better solution.

Could you please confirm if this issue is still relevant to the latest version of the langchainjs repository? If it is, please let the langchainjs team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[andrewnguonly]

I'm running into a similar issue with using the document loaders (e.g. TextLoader) in a Chrome extension. I'm trying to figure out how to get Webpack to build the extension. Will update this thread if I have a solution.

[Charles1713]

This solution above did not work.

After looking closer at the LangChain source code for TextLoader, I found a way to avoid the call to readFile:

if (typeof this.filePathOrBlob === "string") {
  const { readFile } = await TextLoader.imports();
  text = await readFile(this.filePathOrBlob, "utf8");
  metadata = { source: this.filePathOrBlob };
} else {
  text = await this.filePathOrBlob.text();
  metadata = { source: "blob", blobType: this.filePathOrBlob.type };
}

So if we pass in a Blob to the loader instead of a file name, it seems to work:

let infile = this.baseDir + "/TestFile.md";
const fileBlob = await fileToBlob(infile); // custom helper function to read a file as a Blob from Electron app
const fileText = await fileBlob.text();
this.loader = new TextLoader(fileBlob);
const docs = await this.loader.load();

Please let me know if there is a better solution.

I just wanted to thank you for this solution. I'm about to test it later, but it gets rid of my fs error.