search

webrecorder / browsertrix-crawler

Run a high-fidelity browser-based web archiving crawler in a single Docker container
https://crawler.docs.browsertrix.com
GNU Affero General Public License v3.0
637 stars 83 forks source link

Issue creating profile for intranet #705

Open MRLeflei opened 1 week ago

MRLeflei commented 1 week ago

Hi,

I'm getting an error trying to create a profile for our intranet using docker desktop for windows.

I'm using: docker run -p 6080:6080 -p 9223:9223 -v C:\profiles:/crawls/profiles/ -it webrecorder/browsertrix-crawler create-login-profile --interactive --url "https://intranet.local" --dns=XX.XX.XX.XX

"message":"Page Load Failed/Interrupted","details":{"type":"exception","message":"net::ERR_TIMED_OUT at https://intranet.local","stack":"Error: net::ERR_TIMED_OUT at https://intranet.local\n at navigate (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Frame.js:181:27)\n at async Deferred.race (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/util/Deferred.js:33:20)\n at async CdpFrame.goto (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/cdp/Frame.js:147:25)\n at async CdpPage.goto (file:///app/node_modules/puppeteer-core/lib/esm/puppeteer/api/Page.js:563:20)\n at async InteractiveBrowser.startLoad (file:///app/dist/create-login-profile.js:343:13)\n at async main (file:///app/dist/create-login-profile.js:180:9)\n at async file:///app/dist/create-login-profile.js:523:1"}}

I have confirmed that the intranet is reacheble from the container... but there must be something else going on.

Thanks

ikreymer commented 1 week ago

There's not much we can do without more specific repro examples. How did you confirm that the intranet is reachable? You can run docker run -it webrecorder/browsertrix-crawler bash and then try curl to see if it works?