search

commercetest / vitals-scraper

Collect data to assist in analysing Google Play Console Android Vitals
MIT License
6 stars 2 forks source link

Evidence gathering on error #50

Open julianharty opened 5 years ago

julianharty commented 5 years ago

The scraper sometimes fails with an error such as the one below. It'd be useful if the script were able to capture a screnshot and the page contents for post-mortem analysis.

(Note: I've provided some extra log lines to show how the script does its best to recover from the problem.)

Page Error Notification found, retrying
✖ [org.kiwix.kiwixmobile] Getting and writing crash clusters to [/<removed>/myopensourceprojects/android-stability-analysis/data/from-android-vitals/android-crash-clusters-org.kiwix.kiwixmobile_1572966904852.json]
Failed to scrape [org.kiwix.kiwixmobile]
✖ [15:15:56] Failed to scrape after [352794ms]: Error: Error: failed to find element matching selector "section[role=article] .gwt-Label"
    at ElementHandle.$eval (/<removed>/myopensourceprojects/vitals-scraper/node_modules/puppeteer/lib/JSHandle.js:418:13)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
  -- ASYNC --
    at ElementHandle.<anonymous> (/<removed>/myopensourceprojects/vitals-scraper/node_modules/puppeteer/lib/helper.js:108:27)
    at DOMWorld.$eval (/<removed>/myopensourceprojects/vitals-scraper/node_modules/puppeteer/lib/DOMWorld.js:149:21)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
  -- ASYNC --
    at Frame.<anonymous> (/<removed>/myopensourceprojects/vitals-scraper/node_modules/puppeteer/lib/helper.js:108:27)
    at Page.$eval (/<removed>/myopensourceprojects/vitals-scraper/node_modules/puppeteer/lib/Page.js:329:29)
    at Page.<anonymous> (/<removed>/myopensourceprojects/vitals-scraper/node_modules/puppeteer/lib/helper.js:109:23)
    at /<removed>/myopensourceprojects/vitals-scraper/src/Downloader.ts:336:34
    at Generator.next (<anonymous>)
    at fulfilled (/<removed>/myopensourceprojects/vitals-scraper/src/Downloader.ts:4:58)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
ℹ Getting and writing crash clusters to [/<removed>/myopensourceprojects/android-stability-analysis/data/from-android-vitals/android-crash-clusters-org.kiwix.kiwixmobile_1572966904852.json] [2/41] [5%]
Page Error Notification found, retrying
ℹ Getting and writing crash clusters to [/<removed>/myopensourceprojects/android-stability-analysis/data/from-android-vitals/android-crash-clusters-org.kiwix.kiwixmobile_1572966904852.json] [3/41] [7%]
ℹ Getting and writing crash clusters to [/<removed>/myopensourceprojects/android-stability-analysis/data/from-android-vitals/android-crash-clusters-org.kiwix.kiwixmobile_1572966904852.json] [4/41] [10%]