search

webrecorder / browsertrix-crawler

Run a high-fidelity browser-based web archiving crawler in a single Docker container
https://crawler.docs.browsertrix.com
GNU Affero General Public License v3.0
613 stars 79 forks source link

black screen on interactive profile creation #310

Open jswrenn opened 1 year ago

jswrenn commented 1 year ago

I'm attempting to use interactive profile creation. I'm using the snippet suggested in the README verbatim:

docker run -p 6080:6080 -p 9223:9223 -v $PWD/crawls/profiles:/crawls/profiles/ -it webrecorder/browsertrix-crawler create-login-profile --url "https://example.com/"

Unfortunately, when I navigate to http://localhost:9223/ the embedded browser portion of the page is completely black. There are no errors in the console. Any ideas of what might be going on?

jswrenn commented 1 year ago

I'm having the same result using a dedicated VNC client, too. Any recommendations of how to proceed here? I'm attempting to login to Reddit, but the auto-login functionality doesn't work for Reddit.

I'm thinking that my next move is probably to write a custom behavior that completes the login form.

ikreymer commented 1 year ago

Hi @jswrenn - that's really strange, I tried to repro it locally with exact same command and it worked fine! What environment are you deploying in? docker/podman/etc.. and os? Another option is to use headless mode with --headless which will not use VNC, however, you also will need to run crawls with that, which may be fine (I think the headless profile is currently different than non-headless one, though that may be changing in latest chromium)

djhmateer commented 1 year ago

I found that pressing create profile on the grey/blank screen... then running the docker command again.. on the second time the expected screen came up.

docker run -p 6080:6080 -p 9223:9223 -v $PWD/crawls/profiles:/crawls/profiles/ -it webrecorder/browsertrix-crawler create-login-profile --url "https://example.com/"