jshemas / openGraphScraper

Node.js scraper service for Open Graph Info and More!
MIT License
669 stars 105 forks source link

Issue getting details for URLs with 302 Redirections - (node:26923) UnhandledPromiseRejectionWarning: Unhandled promise rejection #149

Closed iamparthaonline closed 2 years ago

iamparthaonline commented 2 years ago

Whenever I try to fetch any details for a URL which has 302 redirections, then the below issue is coming -

image

I tried getting details for the below URL - https://www.youtube.com/c/autocarindia1

I20220811-19:36:31.700(5.5)? options { url: 'https://www.youtube.com/c/autocarindia1' } W20220811-19:36:32.312(5.5)? (STDERR) (node:26923) UnhandledPromiseRejectionWarning: #<Object> W20220811-19:36:32.313(5.5)? (STDERR) (node:26923) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag--unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 10)

I tried adding a try-catch block also still the error persists. Below is the code that I'm trying to run -


const ogs = require("open-graph-scraper");
// need to add Tunnels & Proxies for Safety

const fetchOpenGraphDetails = (options) =>
  new Promise((resolve, reject) => {
    try {
      console.log("options", options);
      ogs(options).then((data) => {
        const { error, result, response } = data;
        console.log("error ------------------------- ", error);
        console.log("result  ------------------------- ", result);
        if (error) {
          resolve({
            success: 0,
            meta: {},
          });
        } else {
          resolve({
            success: 1,
            result,
          });
        }
      });
    } catch (e) {
      console.log("error  -------------------------  :: ", e);
      resolve({
        success: 1,
        result,
      });
    }
  });

export default fetchOpenGraphDetails;
jshemas commented 2 years ago

Hello,

I'm able to see the true error using the code below.

const ogs = require('open-graph-scraper');

const fetchOpenGraphDetails = (options) => new Promise((resolve, reject) => {
  console.log('options', options);
  return ogs(options)
    .then((data) => {
      const { error, result, response } = data;
      console.log('error ------------------------- ', error);
      console.log('result  ------------------------- ', result);
      if (error) {
        resolve({
          success: 0,
          meta: {},
        });
      } else {
        resolve({
          success: 1,
          result,
        });
      }
    })
  .catch((e) => {
    console.log('error  -------------------------  :: ', e);
    resolve({
      success: 0,
      meta: {},
    });
  })
});

fetchOpenGraphDetails({url: 'https://www.youtube.com/c/autocarindia1'});

fetchOpenGraphDetails({url: 'https://ogp.me/'});

Output:

options { url: 'https://www.youtube.com/c/autocarindia1' }
error  -------------------------  ::  {
  error: true,
  result: {
    success: false,
    requestUrl: 'https://www.youtube.com/c/autocarindia1',
    error: 'Exceeded the download limit of 1000000 bytes',
    errorDetails: Error: Exceeded the download limit of 1000000 bytes
        at setOptionsAndReturnOpenGraphResults (/home/node_modules/open-graph-scraper/lib/openGraphScraper.js:92:13)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async /home/node_modules/open-graph-scraper/index.js:29:17
  }
}
options { url: 'https://ogp.me/' }
error -------------------------  false
result  -------------------------  {
  ogTitle: 'Open Graph protocol',
  ogType: 'website',
  ogUrl: 'https://ogp.me/',
  ogDescription: 'The Open Graph protocol enables any web page to become a rich object in a social graph.',
  ogImage: {
    url: 'https://ogp.me/logo.png',
    width: '300',
    height: '300',
    type: 'image/png'
  },
  charset: 'utf8',
  requestUrl: 'https://ogp.me/',
  success: true
}
iamparthaonline commented 2 years ago

Hey Thanks, It was a small syntax issue. Thanks for the help.