jshemas / openGraphScraper

Node.js scraper service for Open Graph Info and More!
MIT License
669 stars 105 forks source link

Can the library keyv be replaced with something else? #181

Closed cibulka closed 1 year ago

cibulka commented 1 year ago

Describe the bug Anytime openGraphScraper is used on Next.js used environment, it creates the following warning in console:

Critical dependency: the request of a dependency is an expression
Import trace for requested module:
./node_modules/keyv/src/index.js
./node_modules/cacheable-request/dist/index.js
./node_modules/got/dist/source/core/index.js
./node_modules/got/dist/source/index.js
./node_modules/open-graph-scraper/dist/lib/utils.js
./node_modules/open-graph-scraper/dist/lib/openGraphScraper.js
./node_modules/open-graph-scraper/dist/index.js

This is a know issue with keyv. The authors of the library claim that it is not a problem and recommend to "close your eyes while the warning's on the screen.", so I don't think this will be fixed.

Unfortunately this issue creates problem on several environments, namely Next.js hosted on Vercel, as described by this comment:

It took me awhile to realize that while I'm using keyv with next.js v12 (and webpack v5) locally, everything would build and work fine, but when deployed to Vercel, any page renders or api calls would throw errors because next.js failed to find the expression-based @keyv/redis import.

To Reproduce Install openGraphScraper on Next.js and call it, either in component or on an api route.

Expected behavior Ideally replace keyv with different dependency.


So to put it shortly - would it be possible to replace keyv with a different library that does not create errors on use?

Thank you!

jshemas commented 1 year ago

This is a issue with cacheable-request / got, so there isn't much I can do. I am working on a rewrite that replaces got with the fetch API. (Which does not use keyv as far as I can tell)

cibulka commented 1 year ago

Cool, thank you! :)

jshemas commented 1 year ago

open-graph-scraper@6.0.0 is now live and no longer uses keyv.