search

EFForg / badger-sett

Automated training for Privacy Badger. Badger Sett automates browsers to visit websites to produce fresh Privacy Badger tracker data.
https://www.eff.org/badger-pretraining
MIT License
121 stars 15 forks source link

Failed to decode response from marionette #8

Closed ghostwords closed 6 years ago

ghostwords commented 6 years ago

This happened at the end of a crawl. All sites following the "failed to decode response" message failed; we should catch this and restart what needs to be restarted.

2018-06-15 00:52:11,847 visiting flavors.me
2018-06-15 00:52:21,768 visiting fastcodesign.com
2018-06-15 00:52:23,197 https://fastcodesign.com/ Failed to decode response from marionette
2018-06-15 00:52:23,197 trying http://fastcodesign.com/
2018-06-15 00:52:23,199 fastcodesign.com Tried to run command without establishing a connection
2018-06-15 00:52:23,199 visiting zippyshare.com
2018-06-15 00:52:23,203 https://zippyshare.com/ Tried to run command without establishing a connection
2018-06-15 00:52:23,203 trying http://zippyshare.com/
2018-06-15 00:52:23,204 zippyshare.com Tried to run command without establishing a connection.
...
2018-06-15 00:52:24,246 visiting realtor.com
2018-06-15 00:52:24,247 https://realtor.com/ Tried to run command without establishing a connection
2018-06-15 00:52:24,247 trying http://realtor.com/
2018-06-15 00:52:24,251 realtor.com Tried to run command without establishing a connection
2018-06-15 00:52:24,251 Scan complete. Saving data...
bcyphers commented 6 years ago

Similar issue: https://github.com/mozilla/geckodriver/issues/1125#issuecomment-356549361

bcyphers commented 6 years ago

For me, this has been resolved by switching to a machine with more memory -- I haven't reproduced since moving to the 4GB droplet.

EDIT: This has now happened on the 4G droplet as well, though I think it was due to multiple docker images running at once. Still trying to figure out a good workaround.

Here's a quick way to reproduce the exception (from https://bugzilla.mozilla.org/show_bug.cgi?id=1401131):

def test_crash(driver):
    driver.set_context("chrome")
    driver.execute_script("""
// copied from crash me simple
Components.utils.import("resource://gre/modules/ctypes.jsm")

// ctypes checks for NULL pointer derefs, so just go near-NULL.
var zero = new ctypes.intptr_t(8);
var badptr = ctypes.cast(zero, ctypes.PointerType(ctypes.int32_t));
var crash = badptr.contents;""")