christopherwk210 / site-scan

CLI for capturing website screenshots, powered by puppeteer.
MIT License
136 stars 13 forks source link

errors in loading and maximum number of urls with command prompt #8

Closed world33 closed 6 years ago

world33 commented 6 years ago

Hi, Lovely and useful code.

I tried this Syntax and urls but it gets stuck (error loading) at one url:

ss --sleep 1 --path D:\screenshots\ -x 1366 -y 768 aou.edu.lb udm.ac.mz ustm.ac.mz unizambeze.ac.mz unipiaget.ac.mz up.ac.mz unilurio.ac.mz sadjad.ac.ir eek.ee lennuakadeemia.ee southvalleyuniversity.com kkmu.ac.zm supershineuniversity.net icuzambia.net chresouniversity.org hvu.edu.vn sgu.edu.vn ucmother.ac.ae imc.gov.ae wus.edu.pk uobs.edu.pk szabmu.edu.pk sbbuvas.edu.pk rmc.edu.pk

Would it be possible to: 1) bypass errors in loading webpages (just go to the next url or add a timeout option)? 2) load the list of urls from a text file? When I paste a large number of urls (up to 1000) in the command prompt there is a maximum number of urls that can be pasted ( I manage to paste up to 572 urls then the last one gets truncated).

If you are not able to fix the issues would you be able to recommend a chrome based script that handles bulk screenshots. I have been searching for ages.

Thanks

christopherwk210 commented 6 years ago

Hey!

4aa33ab09329e65d14435443a2dee9fa74eb7d13 This commit updates things so that if a site fails to load, it will skip to the next item in the list...

905c434a6575d254ad64672966b69eaba25bf3df ...and this commit will allow you to load URLs from a text file 👍

I've updated the npm package as well, so you should be good to update. I'm not going to add a timeout option for now, but if you feel it's necessary feel free to open another issue or make a PR. Closing for now!

world33 commented 6 years ago

Hi Chris, I updated the package but now I get the following error:

Your environment has been set up for using Node.js 10.8.0 (x64) and npm.

C:\Users\Fabio>ss --sleep 1 --path D:\screenshots\ -x 1366 -y 768 aou.edu.lb udm .ac.mz ustm.ac.mz unizambeze.ac.mz unipiaget.ac.mz up.ac.mz unilurio.ac.mz sadja d.ac.ir eek.ee lennuakadeemia.ee southvalleyuniversity.com kkmu.ac.zm supershine university.net icuzambia.net chresouniversity.org hvu.edu.vn sgu.edu.vn ucmother .ac.ae imc.gov.ae wus.edu.pk uobs.edu.pk szabmu.edu.pk sbbuvas.edu.pk rmc.edu.pk

Launching chrome... ? Done.

Loading http://aou.edu.lb... (node:7612) UnhandledPromiseRejectionWarning: Error: Execution context was destr oyed, most likely because of a navigation. at rewriteError (C:\Users\Fabio\AppData\Roaming\npm\node_modules\site-scan\n ode_modules\puppeteer\lib\ExecutionContext.js:144:15) at process._tickCallback (internal/process/next_tick.js:68:7) -- ASYNC -- at ExecutionContext. (C:\Users\Fabio\AppData\Roaming\npm\node_mod ules\site-scan\node_modules\puppeteer\lib\helper.js:144:27) at ExecutionContext.evaluate (C:\Users\Fabio\AppData\Roaming\npm\node_module s\site-scan\node_modules\puppeteer\lib\ExecutionContext.js:58:31) at ExecutionContext. (C:\Users\Fabio\AppData\Roaming\npm\node_mod ules\site-scan\node_modules\puppeteer\lib\helper.js:145:23) at Frame.evaluate (C:\Users\Fabio\AppData\Roaming\npm\node_modules\site-scan \node_modules\puppeteer\lib\FrameManager.js:439:20) at process._tickCallback (internal/process/next_tick.js:68:7) -- ASYNC -- at Frame. (C:\Users\Fabio\AppData\Roaming\npm\node_modules\site-s can\node_modules\puppeteer\lib\helper.js:144:27) at Page.evaluate (C:\Users\Fabio\AppData\Roaming\npm\node_modules\site-scan\ node_modules\puppeteer\lib\Page.js:728:43) at Page. (C:\Users\Fabio\AppData\Roaming\npm\node_modules\site-sc an\node_modules\puppeteer\lib\helper.js:145:23) at takeScreenshot (C:\Users\Fabio\AppData\Roaming\npm\node_modules\site-scan \lib\site-scan.js:149:25) at process._tickCallback (internal/process/next_tick.js:68:7) (node:7612) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1) (node:7612) [DEP0018] DeprecationWarning: Unhandled promise rejections are depre cated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.