search

iulspop / slack-web-scraper

Puppeteer configured to scrape the posts and threads of any channel on Slack.
MIT License
66 stars 19 forks source link

Timeout once navigating to channel #22

Open sionsmith opened 5 months ago

sionsmith commented 5 months ago

Any ideas on this error:

slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:26
            this.#timeoutError = new Errors_js_1.TimeoutError(opts.message);
                                 ^

TimeoutError: Navigation timeout of 30000 ms exceeded
    at new Deferred (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:26:34)
    at Deferred.create (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:68:16)
    at new LifecycleWatcher (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:65:60)
    at CdpFrame.waitForNavigation (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:183:29)
    at CdpFrame.<anonymous> (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:98:27)
    at CdpPage.waitForNavigation (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:623:43)
    at loginAndSaveCookies (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/src/collectData/utils/loginToSlack.js:25:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async loginToSlack (/Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/src/collectData/utils/loginToSlack.js:15:5)
    at async /Users/sionsmith/development/oso/com.github.osodevops/slack-web-scraper/src/collectData/collectData.js:15:3
iulspop commented 5 months ago

It looks like some step in loginToSlack is timing out. Should run it with headless mode set to false to see what is failing.