jshemas / openGraphScraper

Node.js scraper service for Open Graph Info and More!
MIT License
672 stars 107 forks source link

error: 'validator_1.default.isURL is not a function' #233

Closed fabianwohlfart closed 2 months ago

fabianwohlfart commented 3 months ago

We upgraded from 6.5.0 to 6.6.3 and now we get the following error on localhost, didn't test it in prod.

await ogs({ url: 'xyz' })

{ error: true,
  result:
   { success: false,
     requestUrl: 'xyz',
     error: 'validator_1.default.isURL is not a function',
     errorDetails:
      TypeError: validator_1.default.isURL is not a function
          at isUrlValid (.../node_modules/open-graph-scraper/dist/esm/lib/utils.js:39:77)
          at validateAndFormatURL (.../node_modules/open-graph-scraper/dist/esm/lib/utils.js:58:19)
          at setOptionsAndReturnOpenGraphResults (.../node_modules/open-graph-scraper/dist/esm/lib/openGraphScraper.js:28:60)
          at run (.../node_modules/open-graph-scraper/dist/esm/index.js:27:55)
          at fetchOpenGraph (...) },
  response: undefined,
  html: undefined }
jshemas commented 3 months ago

Hello. What version of node are using and what does your tsconfig.json look like?

fabianwohlfart commented 3 months ago

Hey. It's Node 20.15.1 and the tsconfig.json is auto-generated by Nuxt (https://nuxt.com/docs/guide/directory-structure/tsconfig). I rolled back to 6.5.0 and verified that everything works fine there.

jshemas commented 3 months ago

Not sure what Nuxt is. Can you make a example app using Nuxt that demos this problem?

fabianwohlfart commented 3 months ago

A JS / Vue framework. But I am more or less sure that it's not all about Nuxt, as ogs works in 6.5.x and breaks in 6.6.x. Here is a minimal repro: https://github.com/fabianwohlfart/test__ogs

./app.vue fetches /api/ogs which is located in ./server/api/ogs /api/ogs is doing await ogs({ url: "https://nytimes.com" }) and failing due to the error above.

(FYI i am away until next thursday, you might not hear back quickly from me)

Thanks!

jshemas commented 2 months ago

Hello! Thank you for the full example! It looks like there is some issues with validator.js in ESM. Fix is live in open-graph-scraper@6.7.0. Let me know if you have any more issues.

Sydney-o9 commented 2 months ago

@jshemas watch out - I think your fix is triggering issues with TypeScript

image

The issue arises because the code is trying to mix export default with export =, which is not allowed in TypeScript. In a module, you cannot use both export default and export = together.

We have had to downgrade to prevent this issue:

image
jshemas commented 2 months ago

@Sydney-o9 That is not related to this issue. Please open a new issue in the future.

Sydney-o9 commented 2 months ago

I didn't know it was unrelated - I have created an issue https://github.com/jshemas/openGraphScraper/issues/237

fabianwohlfart commented 2 months ago

Thanks for the quick fix!