laurengarcia / url-metadata

NPM module: Request a url and scrape the metadata from its HTML using Node.js or the browser.
https://www.npmjs.com/package/url-metadata
MIT License
166 stars 44 forks source link

Promise neither resolves nor rejects #3

Closed bb-work closed 5 years ago

bb-work commented 6 years ago

I'm using url-metadata to retrieve metadata about URLs that are gotten from another search. Sometimes, neither my success nor failure functions are called, and I don't understand why.

Example "bad" url: http://sartma.com/art_13825.html

for (var i = 0; i < url_list.length; i++){
                        winston.debug(url_list[i]['MentionIdentifier']);
                        urlMetadata(url_list[i]['MentionIdentifier']).then(
                            (metadata) => {
                                var title = metadata.title;
                                var author = metadata.author;
                                var description = metadata.description;
                                var keywords = metadata.keywords;
                                var source = metadata.source;

                                var image;

                                if (metadata.image)
                                {
                                    if (metadata.image.substring(0,2) == '//')
                                    {
                                        image = metadata.image.replace('//','');
                                    } else
                                    {
                                        image = metadata.image.replace('https','http');
                                    }

                                }

                                var url = metadata.url;

                                events.push([image, title, url, author, description, keywords, source])

                                winston.debug(image, title, url, author, description, keywords, source);
                        },
                        (error) => {
                            winston.error("URL Metadata failure: " + error);
                        });

                    }
bb-work commented 6 years ago

I found the issue. The site is returning a statusCode of 403. index.js line 33 should be ||, not &&:

if (!response.statusCode && response.statusCode !== 200) { return dfd.reject({Error: 'response code ' + response.statusCode}) }

laurengarcia commented 6 years ago

Thanks for catching this, sorry for being slow. Will patch asap.

kulinsj commented 6 years ago

Running into this as well. Would be great if this were merged to correct this https://github.com/LevelNewsOrg/url-metadata/pull/4

laurengarcia commented 5 years ago

Merged and updated on npm: https://www.npmjs.com/package/url-metadata