extractus / article-extractor

To extract main article from given URL with Node.js
https://extractor-demos.pages.dev/article-extractor
MIT License
1.46k stars 132 forks source link

Crashing on start with npm run dev #359

Closed alazsengul closed 11 months ago

alazsengul commented 11 months ago

Hi, I'm getting the below crash when running npm run dev. Am I following the tutorial incorrectly? Thanks in advance!

[nodemon] starting `ts-node src/index.ts`
/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/index.js:851
            return old(m, filename);
                   ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/alazsengul/Desktop/stuff/backend/node_modules/@extractus/article-extractor/src/main.js from /Users/alazsengul/Desktop/stuff/backend/src/modules/url.ts not supported.
Instead change the require of main.js in /Users/alazsengul/Desktop/stuff/backend/src/modules/url.ts to a dynamic import() which is available in all CommonJS modules.
    at require.extensions.<computed> [as .js] (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/index.js:851:20)
    at Object.<anonymous> (/Users/alazsengul/Desktop/stuff/backend/src/modules/url.ts:50:27)
    at m._compile (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/index.js:857:29)
    at require.extensions.<computed> [as .ts] (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/index.js:859:16)
    at Object.<anonymous> (/Users/alazsengul/Desktop/stuff/backend/src/index.ts:49:13)
    at m._compile (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/index.js:857:29)
    at require.extensions.<computed> [as .ts] (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/index.js:859:16)
    at phase4 (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/bin.js:466:20)
    at bootstrap (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/bin.js:54:12)
    at main (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/bin.js:33:12)
    at Object.<anonymous> (/Users/alazsengul/Desktop/stuff/backend/node_modules/ts-node/dist/bin.js:579:5) {
  code: 'ERR_REQUIRE_ESM'
}
[nodemon] app crashed - waiting for file changes before starting...

My import looks like the following:

import { ArticleData, extract } from "@extractus/article-extractor";
export async function urlScrapePage(url: string): Promise<ArticleData> {
  try {
    const article = await extract(url);
    return article;
  } catch (err) {
    console.error(err);
    return null;
  }
}

And my package looks like the following:

{
  "name": "backend",
  "version": "1.0.0",
  "description": "",
  "main": "index.ts",
  "scripts": {
    "dev": "nodemon --exec ts-node src/index.ts",
    "test": "echo \"Error: no test specified\" && exit 1",
    "build": "tsc",
    "start": "node ./dist/index.js"
  },
  "engines": {
    "node": "18.0.0"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@types/express": "^4.17.17",
    ...
    "@types/node": "^20.4.2",
  },
  "dependencies": {
    "@extractus/article-extractor": "^8.0.1",
    ...
    "typescript": "^5.1.6",
  }
}
alazsengul commented 11 months ago

I saw that v7.3.0 stopped supporting the CommonJS version.

So I instead installed v7.2.18 and that seemed to work: npm i @extractus/article-extractor@7.2.18.

I'm unfamiliar with CommonJS vs. ES modules, so maybe I'm setting up my project incorrectly? I'm using this package as a helper function for an Express API server.

alazsengul commented 11 months ago

Actually update, it doesn't work when I push it to production. Only on development environment.

ndaidong commented 11 months ago

@alazsengul it seems your platform still needs CJS support, similar to this issue: extractus/oembed-extractor#177

Please try the version v7.3.1 I've just released.

alazsengul commented 11 months ago

@ndaidong Thank you for the quick response + help! It now works.

Out of curiosity, for CommonJS support is that something I need to support in my project? It's just a boilerplate Express server. Or is CommonJS something that needs to be supported by the environment I'm deploying the project in (in this case it's Render)?

ndaidong commented 11 months ago

@alazsengul if you want to update your project to run with ESM, you have to:

I've created a simple service with @extractus/article-extractor v8.0.1 on Render and it works. In their documentation, I found that Render platform support 14.17.0 as default. That's good news because since this version, you can use ESM without any problem.

Screenshot from 2023-07-24 10-13-00

alazsengul commented 11 months ago

Gotcha - thank you so much for the explanation!