could some additional settings improve resource usage during training?

[jawz101]

Since we're just trying to observe certain types of info per domain, could any of these settings improve the performance during training or reduce the resources needed to move through the training without compromising what domains are caught?

some permissions.default.image = 2 --disable images browser.display.use_document_fonts = 0 --disable fonts gfx.downloadable_fonts.enabled = false --disable fonts dom.serviceWorkers.enabled = false -- disable creation of service workers media.peerconnection.enabled = false -- disable webrtc security.OCSP.enabled = 0 --disable querying csp for every cert check webgl.disabled = true --disable webgl layout.spellcheckDefault = 0 --disable spellchecker network.dns.disablePrefetch = true --disable dns prefetching since we aren't clicking on everything during the scan

curious: network.cookie.maxPerHost = 32 dom.workers.maxPerDomain = 48 network.http.max-connections = 48 javascript.options.mem.max = 16384 network.websocket.max-connections = 20

another question increase network.cookie.maxNumber ?

misc. anything with the words network.*max or javascript.options that may make it hang on a page longer than it needs to catch what needs to be caught

disable some things related to media or the max size of things / max retries or timeouts of things. I dunno

[ghostwords]

Hi, thanks for the suggestions!

As you wrote, we don't want to disable anything that would make our browser significantly different from a typical browser. For example, disabling service workers is a no-go probably, since popular websites rely on or even entirely depend on service workers.

We ran into significant slowdowns after updating to Chrome 85, which seem to have gotten addressed by disabling hardware accelerated rendering.

Do you know if any of the above tweaks should produce non-trivial speedups?

[jawz101]

um... I just had a thought. If I'm reading it correctly, It looks like Mozilla's little Selenium privacy testing benchmark thingy, OpenWPM, does disable some prefs. I think it is similarly-oriented automated web crawler tool they've made for privacy research.

https://github.com/mozilla/OpenWPM/blob/master/openwpm/deploy_browsers/configure_firefox.py

It looks like a lot of things they turn off deal with disabling phoning back to Mozilla (auto updates, search boxes, telemetry, crash reporting, etc.) and then some of the prefetching and querying ocsp responders:

    # Predictive Actions / Prefetch
    prefs["network.predictor.enabled"] = False
    prefs["network.dns.disablePrefetch"] = True
    prefs["network.prefetch-next"] = False
    prefs["browser.search.suggest.enabled"] = False
    prefs["network.http.speculative-parallel-limit"] = 0
    prefs["keyword.enabled"] = False  # location bar using search
    prefs["browser.urlbar.userMadeSearchSuggestionsChoice"] = True
    prefs["browser.casting.enabled"] = False

[ghostwords]

I think a lot of those Firefox overrides are specific to OpenWPM.

The gist I think is that OpenWPM uses a proxy to monitor traffic and defaults to stateless crawling (browser restarts for every website). So most of the overrides are there to remove the unwanted Mozilla network traffic on Firefox startup.

Badger Sett doesn't use a proxy to capture traffic at this time and isn't meant to restart the browser much.

[jawz101]

thanks. Well, I'll close this for now