Open spencermize opened 1 month ago
To resolve the ERR_REQUIRE_ESM
error when importing FireCrawlLoader
from @langchain/community/document_loaders/web/firecrawl
in a NextJS nodejs
route, you need to ensure that your Node.js environment supports ES modules. This can be done by setting the type
field in your package.json
to "module"
and using the .mjs
file extension for your module files.
Here is how you can modify your setup:
Update package.json
:
{
"type": "module"
}
Rename your file to use the .mjs
extension:
Rename your route file from yourfile.js
to yourfile.mjs
.
Use import
syntax in your .mjs
file:
import { FireCrawlLoader } from "@langchain/community/document_loaders/web/firecrawl";
const loader = new FireCrawlLoader({
url: "https://firecrawl.dev", // The URL to scrape
apiKey: process.env.FIRECRAWL_API_KEY, // Optional, defaults to `FIRECRAWL_API_KEY` in your env.
mode: "scrape", // The mode to run the crawler in. Can be "scrape" for single urls or "crawl" for all accessible subpages
params: {
// optional parameters based on Firecrawl API docs
// For API documentation, visit https://docs.firecrawl.dev
},
});
const docs = await loader.load();
By following these steps, you should be able to resolve the ERR_REQUIRE_ESM
error and successfully import and use FireCrawlLoader
in your NextJS nodejs
route [1].
To continue talking to Dosu, mention @dosu.
cc @ericciarla have you seen anything like this before?
Update: if I use "type": "module"
in my package.json
, the problem disappears. Is that a requirement of Langchain?
@spencermize no but some third-party packages may only support one or the other natively - we may be able to shim it in. Will ping their team as well.
We just added an issue for this!
You rock - thanks! In the immediate term, we're good with running "type": "module"
, so we can close this ticket unless you want to keep it open.
@jacoblee93 @bracesproul We just pushed commonjs support for it. Version 0.0.31. 🚀
Edit: version 0.0.35 fixes not 31
Ah awesome! Will push a new version with the updated dep. We should also relax it to use ~
I think
@jacoblee93 Apologies, version 0.0.35* fixes it (not 31)! Thank you!
Checked other resources
Example Code
import { FireCrawlLoader } from '@langchain/community/document_loaders/web/firecrawl'
Error Message and Stack Trace (if applicable)
Description
The above import fails in NextJS
14.2.5
, when imported via a route whose runtime isnodejs
.Here's my
tsconfig
's potentially applicable section:Nothing special in our
next.config.js
System Info
Mac Node 20.11.1 Yarn 4.0.2