webrecorder / warcio.js

JS Streaming WARC IO optimized for Browser and Node
MIT License
30 stars 6 forks source link

Trouble requiring module in Node.js #69

Open halmos opened 1 month ago

halmos commented 1 month ago

Requiring warcio seems to result in ES module require errors. eg:

const warcio = require("warcio");

Results in the error require() of ES Module node_modules/base32-encode/index.js from node_modules/warcio/dist/index.cjs not supported.

I also tried

const warcio = require("warcio/node");

but got the error require() of ES Module node_modules/tempy/index.js from /node_modules/warcio/dist/node/index.cjs not supported.

I'm attempting to use warcio in a project with typescript using the module: "CommonJs" setting.

An idea what might be the underlying issue?

halmos commented 2 weeks ago

FWIW - I was able to find a workaround by pinning the version of base32-encode to v1.2 (before it was converted to an ESM-only module). This may be a possible solution for warcio as well, since I don't believe v2 introduced any new features.

maphe commented 6 days ago

I did not have quite exactly the same error message but similar ESM issue so I figured I may as well share my workaround here, in case anybody's interested (although be aware you lose the benefits of typing by doing so):

const Warcio = Function('return import("warcio")')()
const { WARCParser } = await Warcio

//... then you can do
const parser = new WARCParser(response.body)
const record = await parser.parse()
console.log(await record.contentText())