akiessling / urlscanner

0 stars 0 forks source link

Handle PDFs correctly #12

Open regniets opened 5 years ago

regniets commented 5 years ago

In headless mode, PDFs will lead to net::ERR_ABORTED https://github.com/GoogleChrome/puppeteer/issues/830 should be excluded from crawling.