search

BrowserWorks / Waterfox

The official Waterfox 💧 source code repository
https://www.waterfox.net
Other
3.72k stars 334 forks source link

AdNauseam, Findx Privacy Control, Ghostery, HTTPS Everywhere, Nano Adblocker, Privacy Badger and uBlock Origin are bugged by single-process Waterfox #580

Closed grahamperrin closed 6 years ago

grahamperrin commented 6 years ago

Observations, steps

Single-process. With any of these enabled:

– some types of navigation, from one URL to another, are bugged.

Most noticeable with Twitter, for example:

  1. go to https://twitter.com/freebsd
  2. overtype freebsd with waterfoxproject
  3. →
  4. Return or Enter

Expected

Waterfox Web Browser (@Waterfoxproject) | Twitter

– in the title bar (if a title is allowed there); in the tab; and in (e.g.) the Task Manager of KDE

Actual result

2018-05-23 22 48 29 waterfox

Workaround

  1. Control-L then Return or Enter

– or reload.

Thoughts

For around a week (beginning ~2018-05-16) I imagined that the issue arose around the time of testing then releasing Waterfox 56.2.0. This evening I proved myself wrong by quickly reproducing the issue with 56.1.0.

It's possible that all three extensions share the same bugged code, however this issue does feel regressive to me.

Not comparably bugged

Disconnect 5.18.21

Malwarebytes Browser Extension BETA 1.0.21

jbeich commented 6 years ago

Only FreeBSD package? Can you bisect downstream commits down, starting with Firefox 56.0.2? Among ~250 patches in 56.1.0 I may have accidentally backported something that wasn't compatible.

Alternatively, try building Waterfox outside of ports:

$ pkg install python27 binutils git
$ git clone https://github.com/MrAlex94/Waterfox waterfox
$ cd waterfox
$ echo "export COMPILER_PATH=/usr/local/bin" >>.mozconfig
$ echo "ac_add_options --disable-debug-symbols" >>.mozconfig
$ ./mach bootstrap # select Firefox for Desktop
$ ./mach build
$ ./mach run
grahamperrin commented 6 years ago

If it helps to diagnose the issue:

For example, after showing Diigo Toolbar:

2018-05-23 22 49 00 waterfox

$ pkg info waterfox | grep Version
Version        : 56.2.0.13_1
$ date ; uname -v
Wed 23 May 2018 20:47:20 BST
FreeBSD 12.0-CURRENT #2 r333587: Mon May 14 01:59:02 BST 2018     root@momh167-gjp4-hpelitebook8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC

Notes

Partly crossing paths with initial responses …

Only FreeBSD package?

No. The reproduction with 56.1.0 was on a Mac.

I can easily get earlier .dmg files to tell whether truly there's a regression, but I'll probably not do so until Thursday evening (late night Wednesday here). I'll aim for 56.0 next unless anyone suggests a different point for comparison.

Opening post steps to reproduce are clean and reproducible.

I have not attempted to reproduce the blankness symptom (the screenshot in this comment) with anything other than my non-clean 'everyday' profile on FreeBSD-CURRENT.

grahamperrin commented 6 years ago

Late night afterthoughts

A

the blankness

Without looking at any console, I guess it's simply symptomatic of there being no content.

B

Meta, tracking: #538

grahamperrin commented 6 years ago

Maybe worth noting

It's not enough to disable uBlock Origin for e.g. Twitter; the issue will persist until uBlock Origin is entirely disabled.

A brief comparison with Firefox

Result

Not reproducible.

grahamperrin commented 6 years ago

For Firefox with e10s disabled, https://bugzilla.mozilla.org/show_bug.cgi?id=1348497#c38 (2018-01-03) was RESOLVED INVALID:

With Firefox 57, all add-ons are WebExtensions and e10s compatible.

I don't intend to test with the extensions or scenarios that featured in Mozilla bug 1348497.

Instead, the brief comparison with Firefox (above) raises hope that there is a commit (or set of commits) – somewhere between what we already have with Waterfox, and what's for Firefox 60.0.1,1 – that will allow reliable use of extensions such as uBlock Origin with a future single-process Waterfox.

I'll test Findx Privacy Control and Privacy Badger with single-process Firefox 60.0.1,1 …

grahamperrin commented 6 years ago

@NoEscape hi, your nickname is familiar (I can't recall why) and this part of your https://github.com/gorhill/uBlock/issues/3200#issue-271273600 is eye-catching:

  • Browser/version: Firefox 56.0.4

https://www.mozilla.org/firefox/56.0.4/releasenotes/ is (as expected) 404 not found and Waterfox 56.0.4 is outdated but existent … at first I guessed that you had used Waterfox 56.0.4 but then I saw the 32-bit OS so that's a bad guess.

In any case …

Single-process Firefox Quantum

@NoEscape please, would you be willing to test whether the issue with/affecting uBlock Origin is reproducible with, say, single-process Firefox 60.0.2? (No longer applicable, given https://github.com/MrAlex94/Waterfox/issues/580#issuecomment-395991127 below.)

  1. about:config?filter=browser.tabs.remote.force-disable
  2. if necessary, create the boolean browser.tabs.remote.force-disable and make it true then quit Firefox.

Single-process is edge case for Firefox Quantum but I'll be interested in your finding.

Here, whilst I do not yet have 60.0.2 I can not reproduce the issue with 60.0.2 (64-bit) in single-process mode with any of the following (tested individually, not all three together): 

$ pkg info firefox | grep Version
Version        : 60.0.2,1
$ date ; uname -v
Fri  8 Jun 2018 18:27:05 BST
FreeBSD 12.0-CURRENT #3 r334382: Thu May 31 01:37:12 BST 2018     root@momh167-gjp4-hpelitebook8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
grahamperrin commented 6 years ago

@ghostwords hi, tl;dr I wonder whether you can help us to identify a Mozilla commit that can be ported to Waterfox 56.x to fix this Waterfox issue, which affects Privacy Badger (and Findx Privacy Control and uBlock Origin).

Additional information

Various site breakages (Wordpress Admin, eBay) in Firefox · Issue #1295 · EFForg/privacybadger:

Mozilla bug 1373776 was RESOLVED WONTFIX.

Privacy Badger issue 1438 was closed following an affirmative comment from @arthurlogilab re:

… firefox 55.0.1 on ubuntu zesty …

Privacy Badger issue 1438 began with @adimascio giving the https://www.logilab.org/ example. Here now with single-process Waterfox 56.2.0.31_3 on FreeBSD-CURRENT:

– as originally described by @adimascio,

… the url changes in the URL navigation bar but the page content doesn't update. …

With Privacy Badger disabled, sites such as Twitter and Logilab.org do work.

ghostwords commented 6 years ago

Sorry, I don't have any additional insight beyond what I wrote in https://bugzilla.mozilla.org/show_bug.cgi?id=1373776#c18

grahamperrin commented 6 years ago

@ghostwords thanks for the speedy reply and pointer.

@br-privacore hi, re: the tl;dr at https://github.com/MrAlex94/Waterfox/issues/580#issuecomment-395853575 above I wonder whether you, too, might help to identify a Mozilla commit.

With Findx Privacy Control enabled, https://www.logilab.org/ and Twitter are troublesome. With the extension disabled, the trouble ceases.

The Twitter and Logilab.org cases are not reproducible (for me) with Firefox 60.0.2 in single-process mode, so I assume that a commit was made somewhere between Firefox 56.0.2 (upon which Waterfox 56.0 was based) and Firefox 60.

br-privacore commented 6 years ago

@grahamperrin : Findx Privacy Control is based on uBlock Origin.

I'm not sure how we can help, as we only test against the few latest releases of each supported browser.

Only thing I can think of is to try every Firefox release between 56 and 60 (https://ftp.mozilla.org/pub/firefox/releases/) and see when the bug disappears, to narrow down the number of patches to investigate.

Did you have anything else in mind?

grahamperrin commented 6 years ago

@br-privacore thanks. I'll go for 57, 58 and 59 on a Mac; the .dmg files will make this easy enough.

… Did you have anything else in mind?

Only the possibility of someone already knowing either (a) the release point at which the bug disappeared; or (b) ideally, the relevant commit(s).

I'll follow up.

grahamperrin commented 6 years ago

Towards identifying the fix

Testing a Firefox profile with browser.tabs.remote.force-disable true and the https://www.logilab.org/ example:

https://ftp.mozilla.org/pub/firefox/releases/58.0b3/ includes files modified on 2017-11-14.

https://wiki.mozilla.org/Releases/Firefox_58/Test_Plan

https://wiki.mozilla.org/Releases/Firefox_58/Test_Plan#Beta_3 refers to:

From https://wiki.mozilla.org/Releases/Firefox_58/Test_Plan/Beta/3#Bug_fix_verification:

Note: All verified Firefox 58 bug fixes are listed in https://goo.gl/fktLi3

jbeich commented 6 years ago
  • Firefox 58.0b3 is not bugged

Beta 3 usually marks the end of Nightly phase for a given milestone. Try using mozregression to bisect every build of Nightly 58 phase: from 2017-09-28 to 2017-11-14.

grahamperrin commented 6 years ago

OK, I knew this wouldn't work, but I had to try it: 

$ mozregression --find-fix --good 2017-11-14 --bad 2017-09-28
**********
You should use a config file. Please use the --write-config command line flag to help you create one.
**********

 0:00.30 ERROR: mozregression supports linux, mac and windows but your os is reported as 'bsd'.

:-) so I reckon, it's time for me to wipe the internal HDD on a nearby MacBookPro8,2 and Boot Camp it with Sierra and a 64-bit Windows (10, if the 2011 Mac hardware will take it).

(I could continue with the Sierra that's currently used, but it's on an external HDD and I suspect that USB 2.0 will make things such as mozregression feel unreasonably slow.)

grahamperrin commented 6 years ago

Also bugged:

– maybe no longer available from addons.mozilla.org but it's mentioned at https://www.reddit.com/comments/8oyy68/-/e07cmvt/?context=2 and does cause e.g. https://www.logilab.org/ to not work with single-process Waterfox 56.2.0.31_5.

mozregression

I haven't found time yet, sorry. (Catching up on sleep; Boot Camp Assistant on the MacBookPro8,2 is limited to Windows 7 (no go with a Microsoft .iso for Windows 10); and so on.)

grahamperrin commented 6 years ago

Below,

The initial good/bad range seems incorrect.

sh-3.2$ date ; sw_vers ; whoami
Thu 14 Jun 2018 07:20:26 BST
ProductName:    Mac OS X
ProductVersion: 10.9.5
BuildVersion:   13F1911
bbsadmin-l
sh-3.2$ mozregression --write-config

You should configure a persist directory, where to put downloaded build files to reuse them in future bisections.
I recommend using /Users/bbsadmin-l/.mozilla/mozregression/persist. Leave blank to use that default. If you really don't want a persist dir type NONE, else you can just define a path that you would like to use.
persist: 
persist: /Users/bbsadmin-l/.mozilla/mozregression/persist

You should choose a size limit for the persist dir. I recommend you to use 20.0 GiB, so leave it blank to use that default. Else you can type NONE to not limit the persist dir, or any number you like (a GiB value, so type 0.5 to allow ~500 MiB).
persist-size-limit: 
persist-size-limit: 20.0

Config file /Users/bbsadmin-l/.mozilla/mozregression/mozregression.cfg written.
Note you can edit it manually, and there are other options you can configure. See http://mozilla.github.io/mozregression/documentation/configuration.html.
sh-3.2$ mozregression --find-fix --good 2017-11-14 --bad 2017-09-28
 0:41.97 INFO: Testing good and bad builds to ensure that they are really good and bad...
 0:41.97 INFO: Downloading build from: https://archive.mozilla.org/pub/firefox/nightly/2017/11/2017-11-14-10-00-42-mozilla-central/firefox-59.0a1.en-US.mac.dmg
===== Downloaded 100% =====
 0:52.44 INFO: Running mozilla-central build for 2017-11-14
 2:17.64 INFO: Launching /private/var/folders/l_/j60gks_9021761h8ktq4w4qh0000gn/T/tmpNVJFsc/FirefoxNightly.app/Contents/MacOS/firefox
 2:17.64 INFO: Application command: /private/var/folders/l_/j60gks_9021761h8ktq4w4qh0000gn/T/tmpNVJFsc/FirefoxNightly.app/Contents/MacOS/firefox -foreground -profile /var/folders/l_/j60gks_9021761h8ktq4w4qh0000gn/T/tmp_dND4J.mozrunner
 2:17.92 INFO: application_buildid: 20171114100042
 2:17.92 INFO: application_changeset: e1d7427787f7a26983c92ea1a1ac99eb863edd6c
 2:17.92 INFO: application_name: Firefox
 2:17.92 INFO: application_repository: https://hg.mozilla.org/mozilla-central
 2:17.92 INFO: application_version: 59.0a1
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry' or 'exit' and press Enter): good
15:55.10 INFO: Using local file: /Users/bbsadmin-l/.mozilla/mozregression/persist/2017-09-28--mozilla-central--firefox-58.0a1.en-US.mac.dmg (downloaded in background)
15:55.12 INFO: Running mozilla-central build for 2017-09-28
17:11.47 INFO: Launching /private/var/folders/l_/j60gks_9021761h8ktq4w4qh0000gn/T/tmpBNXAe7/FirefoxNightly.app/Contents/MacOS/firefox
17:11.47 INFO: Application command: /private/var/folders/l_/j60gks_9021761h8ktq4w4qh0000gn/T/tmpBNXAe7/FirefoxNightly.app/Contents/MacOS/firefox -foreground -profile /var/folders/l_/j60gks_9021761h8ktq4w4qh0000gn/T/tmpQzEamm.mozrunner
17:11.57 INFO: application_buildid: 20170928220658
17:11.57 INFO: application_changeset: 307a7a34013060a6a1e87dfbb911f058d0781a2e
17:11.57 INFO: application_name: Firefox
17:11.57 INFO: application_repository: https://hg.mozilla.org/mozilla-central
17:11.57 INFO: application_version: 58.0a1
Was this nightly build good, bad, or broken? (type 'good', 'bad', 'skip', 'retry' or 'exit' and press Enter): good
49:26.12 ERROR: Build was expected to be bad! The initial good/bad range seems incorrect.
sh-3.2$ date
Thu 14 Jun 2018 08:20:13 BST
sh-3.2$
jbeich commented 6 years ago

@grahamperrin, try specifying an earlier start date (--bad in your case). Refer to calendar to check milestone (see central). For one, 2017-06-12 would be start of 56.0.

jbeich commented 6 years ago

In other words, you need to find a date of Nightly build that's affected by the issue at hand. mozregression cannot bisect the range if both of the ends are good or both are bad.

grahamperrin commented 6 years ago

2018-06-14T18 bisection log.txt

From the log: 

108:27.47 INFO: No more inbound revisions, bisection finished.
108:27.47 INFO: First good revision: d7c3a31373a61c8da5e5e615be725738c59464cd
108:27.47 INFO: Last bad revision: d1d7b2a2bad7b7e91c085c9eb0b3acca29123fcc
108:27.47 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=d1d7b2a2bad7b7e91c085c9eb0b3acca29123fcc&tochange=d7c3a31373a61c8da5e5e615be725738c59464cd

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=d1d7b2a2bad7b7e91c085c9eb0b3acca29123fcc&tochange=d7c3a31373a61c8da5e5e615be725738c59464cd refers to:

RESOLVED FIXED in Firefox 58

I did look at 1396395 a few days ago but I imagined that it was unrelated, because with the given steps to reproduce:

  1. Open http://sagamusix.untergrund.net/fileupload.htm in Firefox 52 ESR.
  2. Select a small file and hit the submit button.

– I could not produce a crash in Waterfox 56.2.0.

The changesets:

jbeich commented 6 years ago

Thanks. Can you try Waterfox 56.2.0.53 on FreeBSD? It already includes #623.

FWIW, bisecting on FreeBSD amounts to building each step from source while applying bustage fixes. Due to time-consuming nature of babysitting build to check for bustage I rarely do it outside of critical issues. And for anyone else the process requires experience with Firefox development workflow.

grahamperrin commented 6 years ago

@jbeich big thanks. If I don't receive a response to https://github.com/MrAlex94/Waterfox/pull/623#issuecomment-397506825 tomorrow I'll close this issue 580 as fixed (subject to merge of the PR).

@Hainish hi, FYI HTTPS Everywhere is also amongst the products that are bugged by single-process.

jbeich commented 6 years ago

@grahamperrin, unnecessary. When #623 is merged this issue would be automatically closed. Try hovering mouse pointer over Fixes keyword there.

grahamperrin commented 6 years ago

@jbeich you're right. I shouldn't encourage premature closure … just, I sort of wanted one more thing to appear ☑ or closed above and below the line at https://github.com/MrAlex94/Waterfox/issues/538#issuecomment-389295921

grahamperrin commented 6 years ago

@dhowe hi, FYI your AdNauseum is also amongst the products that are bugged by single-process.