jshemas / openGraphScraper

Node.js scraper service for Open Graph Info and More!
MIT License
643 stars 102 forks source link

Why the error with a generic 400/500 error message? #190

Closed WaylonOKC closed 1 year ago

WaylonOKC commented 1 year ago

v5 would return the actual error returned by the website.

For example, if you try to scrap https://www.congress.gov/bill/117th-congress/house-bill/2617/text then you will get a 403-Forbidden error.

v6 now catches all 4xx and 5xx errors and returns a generic "Server has returned a 400/500 error code" which is not helpful/useful.

Why hide the actual error?

jshemas commented 1 year ago

Hello,

Fixed in open-graph-scraper@6.2.1

This will now return:

result: {
  success: false,
  requestUrl: 'https://www.congress.gov/bill/117th-congress/house-bill/2617/text',
  error: '403 Forbidden',
  errorDetails: Error: 403 Forbidden
      at requestAndResultsFormatter (/openGraphScraper/lib/request.ts:36:17)
      at processTicksAndRejections (node:internal/process/task_queues:95:5)
      at setOptionsAndReturnOpenGraphResults (/openGraphScraper/lib/openGraphScraper.ts:44:32)
      at run (/openGraphScraper/index.ts:26:15)
}
WaylonOKC commented 1 year ago

Thanks!!