search

prolificinteractive / node-html-to-json

Parses HTML strings into objects using flexible, composable filters.
MIT License
120 stars 13 forks source link

scraping web https #8

Open komarudin2211 opened 6 years ago

komarudin2211 commented 6 years ago

i want scraping data website with https type is possible? i try scraping always return empty data.

prolificeric commented 6 years ago

@komarudin2211 It's definitely possible. Once thing I would check for is if the site expects particular headers or other parameters in the request. For instance I've run into a case where a site won't accept requests without a User-Agent header.