geckodriver render youtube fail

[cwchiu]

System

• Version: 0.24.0 ( 2019-01-28)
• Platform: docker container (Debian GNU/Linux 9 (stretch))

FROM joyzoursky/python-chromedriver:3.7

# install geckodriver and firefox

RUN apt-get install -y libdbus-glib-1-2 && \
    GECKODRIVER_VERSION=`curl https://github.com/mozilla/geckodriver/releases/latest | grep -Po 'v[0-9]+.[0-9]+.[0-9]+'` && \
    wget https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-linux64.tar.gz && \
    tar -zxf geckodriver-$GECKODRIVER_VERSION-linux64.tar.gz -C /usr/local/bin && \
    chmod +x /usr/local/bin/geckodriver && \
    rm geckodriver-$GECKODRIVER_VERSION-linux64.tar.gz

RUN FIREFOX_SETUP=firefox-setup.tar.bz2 && \
    apt-get purge firefox && \
    wget -O $FIREFOX_SETUP "https://download.mozilla.org/?product=firefox-latest&os=linux64" && \
    tar xjf $FIREFOX_SETUP -C /opt/ && \
    ln -s /opt/firefox/firefox /usr/bin/firefox && \
    rm $FIREFOX_SETUP

• Firefox: 67.0.2
• Selenium: 3.141.0

Testcase

Firefox Fail

from selenium import webdriver
options = webdriver.FirefoxOptions()
options.add_argument("-headless")
driver = webdriver.Firefox(options=options)
driver.get('https://www.youtube.com/watch?v=wUPPkSANpyo')
driver.save_screenshot('/tmp/ff001.png')
driver.execute_script('return window.chatframe') # None

Chrome Success

from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
options.add_argument("--headless")
options.add_argument("--disable-extensions")
options.add_argument("--disable-notifications")
options.add_argument("--disable-infobars")
options.add_argument("start-maximized")
driver = webdriver.Chrome(options=options)
driver.get('https://www.youtube.com/watch?v=wUPPkSANpyo')
driver.save_screenshot('/tmp/chrome001.png')
driver.execute_script('return window.chatframe') # WebElement

[andreastt]

To me it looks like youtube.com hasn’t finished loading yet by the point the screenshot is taken?

[cwchiu]

i waited for 10 min. but i still get the same screenshot

What should I do?

[andreastt]

And loading the document without a WebDriver session works just fine?

Sorry to keep asking questions, but this issue lacks enough details to be reproducible.

[whimboo]

Try to run Firefox via -screenshot %url% to create a screenshot in headless mode.

[cwchiu]

Ok, I run

got screenshot.png

is this a firefox bug?

[whimboo]

Yes, as it looks like. I did a quick check and actually found https://bugzilla.mozilla.org/show_bug.cgi?id=1412061 which sounds to be related. Feel free to comment over there.

What's interesting is that Marionette causes a similar outcome. I wonder if most of the page content is loaded via AJAX, and as such the page isn't fully displayed yet. Checking with the network panel of dev tool might give an indication here.

[cwchiu]

@whimboo Thanks for your reply.

[whimboo]

Please see the following Firefox bug which I just filed: https://bugzilla.mozilla.org/show_bug.cgi?id=1563720

[whimboo]

Also note that in the WebDriver specification we currently do not have an option for users to wait for every resource to be loaded, but just for DOMContentLoaded and pageshow. So it's not doable with geckodriver right now.

[whimboo]

See also https://github.com/w3c/webdriver/issues/895 and https://github.com/w3c/webdriver/issues/893.

I will reopen this issue until it can be fully fixed.