Any plan to include Transformers.js or ONNX runtime?

[Durisvk]

The libraries listed above are free to use and open source:

• https://github.com/xenova/transformers.js
• https://onnxruntime.ai/

Unlike OpenAI and other services by various cloud providers.

Would be super nice to see an integration with LangchainJS ecosystem.

I'm happy to help if needed, I'm just curious if it'd be something you'd like to explore before I put any effort into it.

[dosubot[bot]]

🤖

Hello,

Thank you for your interest in LangChain and your suggestion to integrate Transformers.js and ONNX runtime. As you've pointed out, these are valuable resources and their integration could potentially enhance the capabilities of LangChain.

From the context provided, it's clear that LangChain does support the integration of external libraries. The framework has optional import entry points for various external libraries and tools, including AWS Lambda, SQL, TensorFlow, Google Vertex AI, Elasticsearch, Puppeteer, Playwright, and others. This indicates that LangChain is designed to be extensible and can integrate with a variety of external libraries.

However, as of now, Transformers.js and ONNX runtime are not listed among the optional import entry points. This doesn't necessarily mean that they can't be integrated, but it does mean that they are not currently supported out of the box.

If you're interested in contributing to the project, I would recommend starting by creating a proposal or a draft pull request to add these libraries as optional import entry points. This would involve writing the necessary code to interface with these libraries and adding them to the list of optional import entry points in the import_constants.ts file.

Before you start, it would be a good idea to discuss this with the LangChain community to ensure that your contribution aligns with the project's goals and roadmap. You can do this by creating an issue on the LangChain GitHub repository to propose your idea and gather feedback.

I hope this answers your question. If you need further clarification, please don't hesitate to ask.

Best regards, Dosu

Sources

• langchain/src/load/import_constants.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Durisvk]

Lol this bot is cool 😄 however I'll wait for authors to confirm before putting any effort into it

[jacoblee93]

We've got #2288 open - am just thinking how to best surface since it doesn't seem to work in a default Node env. Would you be up for taking it over/writing docs?

[Durisvk]

Hi @jacoblee93 I've looked into it, I can fix it in a hacky way (it's a problem with Jest's context isolation) but I want to figure out a proper fix for that

[jacoblee93]

Ah interesting - let me know what you find! Would love to add it, I think WebML stuff is pretty neat

[Durisvk]

Hey @jacoblee93 the solution was to share Float32Array between jest isolated contexts: https://github.com/hwchase17/langchainjs/pull/2487

It's the least hacky solution my research could come up with