search

wikimedia / html-metadata

MetaData html scraper and parser for Node.js (supports Promises and callback style)
MIT License
165 stars 44 forks source link

Receiving HTTPError: 403 from Shopify Sites #78

Closed mbillz closed 11 months ago

mbillz commented 6 years ago

This library is working great in all areas except when we try to scrape a Shopify site. When running the main scrape function the JSON response returned is an error message with string "Receiving HTTPError: 403 from Shopify Sites".

Any guidance? You can reproduce from the Node app we setup at https://metaparser-v2.thesocialpresskit.com by passing a url in as a query argument. A Shopify example of this is https://metaparser-v2.thesocialpresskit.com/?url=https://sincerelyjewelry.com/

Thanks in advance for any help!!

mvolz commented 6 years ago

Have you tried setting a cookie in the options object?

mbillz commented 6 years ago

we ended up moving to a different library. thanks though!