search

BrowserWorks / Waterfox

The official Waterfox 💧 source code repository
https://www.waterfox.net
Other
3.78k stars 337 forks source link

Waterfox treating signed addons as unsigned #936

Closed laniakea64 closed 5 years ago

laniakea64 commented 5 years ago

Describe the bug When installing a signed addon, Waterfox erroneously says it is unverified.

To Reproduce Steps to reproduce the behavior:

new profile

  1. Go to https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/
  2. Click "Add to Firefox" to install uBlock Origin

Expected behavior Install prompt should not have a warning about unverified addon

Screenshots Screen shot 2019-05-04 1

Desktop (please complete the following information):

Additional context This is probably related to Mozilla bug 1548973

hook321 commented 5 years ago

I confirmed that this is caused by Mozilla forgetting that their certificate was going to expire. In Firefox this caused people's addons to get disabled, in Waterfox the effects aren't as serious. https://blog.mozilla.org/addons/2019/05/04/update-regarding-add-ons-in-firefox/

laniakea64 commented 5 years ago

Related Browser Console message -

Signature Verification Error: the signature on this .jar archive is invalid because the digital signature (*.RSA) file is not a valid signature of the signature instruction file (*.SF).
grahamperrin commented 5 years ago

Track https://discourse.mozilla.org/t/-/39845?u=grahamperrin – expect advice from Mozilla in due course.

In the meantime there's a workaround for most cases where an add-on will either not install, or not update:

NB the Wood time example in the overview at https://www.reddit.com/comments/bktabg/-/emmt6qp/

laniakea64 commented 5 years ago

The fix from mozilla-esr60 looks to fix this issue in Waterfox. I'll do a pull request soon.

laniakea64 commented 5 years ago

The impact of this issue on Waterfox is more serious than I originally realised.

Waterfox 56.2.9 can no longer install signed WebExtensions that don't specify an addon ID in their manifest.json. Example: https://addons.mozilla.org/firefox/addon/cookie-quick-manager/

Browser Console message -

addons.xpi  WARN    Download of https://addons.mozilla.org/firefox/downloads/file/1676320/cookie_quick_manager-0.4rc4-an+fx.xpi?src=dp-btn-primary failed: Error: Cannot find id for addon /tmp/tmp-n3i.xpi (resource://gre/modules/addons/XPIInstall.jsm:1643:17) JS Stack trace: loadManifest@XPIInstall.jsm:1643:17

This problem does not exist in a custom Waterfox build with the patches from the pull request.

grahamperrin commented 5 years ago

Example: https://addons.mozilla.org/firefox/addon/cookie-quick-manager

Yep, in my patch-free default profile with 56.2.9 on FreeBSD-CURRENT:

Download failed. Please check your connection.

The add-on downloaded from this site could not be installed because it appears to be corrupt.

No problem with a profile into which I imported the icfix.pem from https://www.velvetbug.com/benb/icfix/

grahamperrin commented 5 years ago

Obliquely related, a Mozilla bug that I began following at the weekend:

– in particular, the response(s) to comment 8. Edge case, and (!) too much to get my head around at the moment, but it might be prudent to discourage end users from messing with their system clock as an interim workaround to armagadd-on-2.0.

grahamperrin commented 5 years ago

:100:

LeeBinder commented 5 years ago

STEPS TO IMPORT THE CERTIFICATE

Different methods to have your extensions act normal again are listed below, starting with the easiest, fastest one with the least footprint and number of clicks - direct import, no extra extension needed, compliantly working for both, Desktop and Android.

RECOMMENDED: "CATCH-ALL Fix", tested with Waterfox Desktop 56.2.9 & Waterfox Android 56.1.0, but should work with any edition and version of Waterfox/ Firefox affected by the armagadd-on 2.0 bug:

  1. Click onto icfix.pem (BASE64 format version credit to Ben B/velvetbug) OR addons-public-intermediate.der (BINARY format version credit to Mozilla) (both formats, binary and base64, have exactly the same effect and are interchangeable)
  2. Click onto OK (no need to check both boxes - you can leave them BLANK when authorizing the certificate it makes! Hint and screenshot credit to grahamperrin): image
  3. To verify if you can install extensions again which are affected by the certificate bug, test-install e.g. Custom UserAgent String

Waterfox DESKTOP 56.2.9 ONLY:

A) Mozilla also provides extensions that import that same certificate into the matching version of Firefox. For instructions about the extension installation, go to Mozilla's 'Disabled Add-on Fix for Firefox 52 - 56' (compatible with Waterfox 56.2.9).

Note that Disabled Add-on Fix for Firefox 47 - 56 (by Mozilla) is NOT immediately working for Waterfox Desktop 56.2.9: "This add-on is not compatible with your version of Firefox" due to Waterfox pretending to be Firefox 57 on AMO in about:config?filter=general.useragent.override (credit to laniakea64 for the hint). It is, however, possible to install it via this direct link (v.1.1.4, 2019-05-14), or (if you want to walk through step-by-step) by right-clicking the greyed out "Add To Firefox.." button > Copy link location, then right-click the address bar > Paste & Go. Via same right-click contextual menu, it can be downloaded manually and be dragged into about:addons, where you should see it active afterwards. In a fresh virgin account, affected Add-ons DO then install.

B) If you prefer to save the certificate to your hard drive to import it for later offline usage:

  1. Download via right ("secondary") click, "Save Link As..": • icfix.pem (BASE64) from either location 1 or location 2 • addons-public-intermediate.der/.crt (BINARY) from either location 1 or location 2
  2. Options/ Preferences, Advanced
  3. View Certificates
  4. Select "Authorities"
  5. Import
  6. Navigate to icfix.pem on your hard drive
  7. Click onto OK to import the certificate

[Credits to megalomaniacs4u for his base fix Addons Fix for Firefox 56.0.2 & older]

C) If you still can't install an extension that should be installable, you could try the steps for Firefox 56.0.2 and older outlines by Ben B:

  1. Open the browser console (Windows: Ctrl + Shift + J | Mac: Cmd + Shift + J) and run the following two lines (copy, paste, enter): 
    Components.utils.import("resource://gre/modules/addons/XPIProvider.jsm");
XPIProvider.verifySignatures();
  2. Test again (Custom UserAgent String)

Waterfox ANDROID 56.1.0 ONLY:

Again I highly recommend you ONLY install the certificate itself as described above (!) because Mozilla made the Extensions for Firefox not for Waterfox and did NOT test them in Waterfox, naturally! In a test run in a new fresh virgin profile, v.1.1.2 of the Extension installed and had the desired effect of affected Add-ons installing afterwards, but the Extension is NOT and can NOT be enabled in Add-ons (even after WF force-quit and re-launch), which rings an alarm bell..

On May 14 2019 Mozilla published v.1.1.4 on their Disabled Add-on Fix for Firefox 47-56 page, but again, think twice if you really voluntarily want to play guinea pig even though there is no need to...

Also please note that in the process of the recent purge of addons.mozilla.org, Mozilla not only removed all legacy Extensions but also Themes, and ONLY left themes for Firefox Android 65.0+!! You might still find an older theme version on addons.mozilla.org which might install, but very most likely it will not have any effect!

I'm confident Alex will also release WF 56.x.x with this fix very soon (as per Alex also for Android "on its way" 🥇) because the release of WF 68 final will still take a while.

BTW, I think this is a worthwhile read for everyone: What you need to know about add-ons in Waterfox 68 - gHacks Tech News

Further links:

reallyuniquename commented 5 years ago

Yeah OK, I was kinda suprised I ran into this issue https://github.com/MrAlex94/Waterfox/issues/936#issuecomment-489872863 issue today since Waterfox already allows unsigned extensions by default and my custom legacy addons work alright and I had no troubles with recent armagadd-on even without new certificate.

Yet I couldn't install one specific modded addon today. I believe only WE addons are affected and fix is pretty simple (via https://github.com/MrAlex94/Waterfox/issues/936#issuecomment-489872863 & https://github.com/MrAlex94/Waterfox/issues/936#issuecomment-490002026 @laniakea64, thanks for the short description) .

Carve-up XPI, open manifest.json and add JSON block:

"applications": {
        "gecko": {
            "id": "<your_addon_id>"
        }
    },

Repack XPI and you are good to go.

By the way this bug forces browser to delete "corrupted" addons.

laniakea64 commented 5 years ago

(Edit: This is in reply to https://github.com/MrAlex94/Waterfox/issues/936#issuecomment-490452890 . I don't understand why this reply is showing above that comment?)

Carve-up XPI, open manifest.json and add JSON block:

"applications": {
        "gecko": {
            "id": "<your_addon_id>"
        }
    },

:+1: This should work if you give it the same addon ID as the AMO version. You should be able to find that ID in about:support

By the way this bug forces browser to delete "corrupted" addons.

I haven't experienced this in Waterfox 56.2.9. Could you please explain more under what circumstances does this happen?

reallyuniquename commented 5 years ago

@laniakea64

This should work if you give it the same addon ID as the AMO version. You should be able to find that ID in about:support

Correct, yes. In case addon was not installed in the first place one should look into mozilla.rsa certificates for proper ID. If addon was initially downloaded from AMO, XPI name is basically its ID.

Could you please explain more under what circumstances does this happen?

  1. Install WebExtension addon that doesn't have id in manifest.json (e.g. Smart HTTPS 0.2.5).
  2. Close Waterfox.
  3. Edit any file inside installed XPI.
  4. Launch Waterfox.

Extension will be gone without any visible notification, Waterfox will move XPI file to trash subfolder inside extensions folder. I believe it will be permanently deleted later but I am not sure.

Alternatively you may download XPI from AMO, edit it and try to install it. Waterfox won't let you do it saying it's "corrupted".

This problem does not exist in a custom Waterfox build with the patches from the pull request.

Are you sure about that? That patch basically installs new certificate and that's it. I've already installed it manually and it's still impossible to install edited addons without touching manifest.json.

My understanding is the issue in the first post and recent armagadd-on-2.0 are totally unrelated, just bad timing.

I wonder how you managed to reply to my post the way it shows above mine 🤔

Edit: Github rearranged our posts (LOL).

grahamperrin commented 5 years ago

Incidentally … without testing the effect of this PR, I find it unnecessary to check any of the three trust options:

image

Simply having the certificate present seems to suffice for e.g. the Cookie Quick Manager case.

Am I missing something?

https://www.reddit.com/r/firefox/comments/bkspmk/-/emkd8qp/ some users of Firefox find the same, no need to check any of the boxes.

LeeBinder commented 5 years ago

Helpful information, all of it - thanks for sharing here!

Three questions:

  1. Which file does icfix.pem get imported into? .. and regarding Waterfox Android:
  2. how can I import the certificate icfix.pem the easiest way (no certificate import option as in Waterfox Desktop)
  3. how can I install a modified xpi (there is no drag-&-drop as in Waterfox Desktop)

Android 7.1.1, rooted

Ibuprophen commented 5 years ago

If you go to the following link...

https://www.reddit.com/r/firefox/comments/bkspmk/addons_fix_for_5602_older/

... And select the link at the very top, it'll install on the Android app as a normal addon.

~Ibuprophen

LeeBinder commented 5 years ago

Thanks, @ibuprophen1 , you just cured some of my headaches .. ;) I never thought the .xpi would have any effect neither in legacy Firefox nor in Waterfox which both don't have or use "normandy", so I sure am surprised that now two oddities I never thought were related to this add-on signing issue are back to normal in my Android Waterfox 👍

Still, as others noted, not all extensions install, like Custom UserAgent String by Linder. Or does it install for anybody, either in Waterfox Android with the .xpi installed, or in Waterfox Mac/ Linux/ Windows with icfix.pem imported?

Would be great if someone could test-install this add-on and feed-back- thanks.

Ibuprophen commented 5 years ago

There's a handful of them that I can't install myself since this nightmare began.

I'm hoping to either figure out or find something to get going at least 2 of the ones I can't install since I do rely on them for a handful of items I do on the Android Browser.

I'm happy to see a good handful of them going so far.

~Ibuprophen

grahamperrin commented 5 years ago

the link at the very top, it'll install on the Android app

Note, the linked file is not a fix for Waterfox 56.2.9.

grahamperrin commented 5 years ago

… not all extensions install, like Custom UserAgent String by Linder. Or does it install for anybody, … in Waterfox Mac/ Linux/ Windows with icfix.pem imported? …

Custom UserAgent String can be added to (desktop) Waterfox 56.2.9 on FreeBSD-CURRENT.

For any add-on that can not be added to a fixed installation of 56.2.9, please raise a separate issue. Thanks

laniakea64 commented 5 years ago

  1. Install WebExtension addon that doesn't have id in manifest.json (e.g. Smart HTTPS 0.2.5).

  2. Close Waterfox.

  3. Edit any file inside installed XPI.

  4. Launch Waterfox.

Extension will be gone without any visible notification, Waterfox will move XPI file to trash subfolder inside extensions folder. I believe it will be permanently deleted later but I am not sure.

Alternatively you may download XPI from AMO, edit it and try to install it. Waterfox won't let you do it saying it's "corrupted".

This problem does not exist in a custom Waterfox build with the patches from the pull request.

Are you sure about that? That patch basically installs new certificate and that's it. I've already installed it manually and it's still impossible to install edited addons without touching manifest.json.

Thanks for the clarification. I think this is unrelated to armagadd-on-2.0, and expected behavior. WebExtensions that are not properly signed have always required addon ID.

I'm surprised you were ever able to get that to work.

I've always had to do this when modifying signed WebExtensions that don't have an ID:

1) Copy the xpi outside of profile folder

2) Delete the signature

3) Add addon ID to manifest.json, as you described

4) Make your modifications

5) Install modified XPI through Waterfox, the normal way, e.g. by drag&drop to tab bar.

laniakea64 commented 5 years ago

Mozilla changed their fix for esr60 - https://hg.mozilla.org/releases/mozilla-esr60/rev/5749f5b42cbf5a972bc8c398ed377977da35dbd2

I don't understand this patch :frowning_face: Does Waterfox need to follow suit?

reallyuniquename commented 5 years ago

@laniakea64

I think this is [...] expected behavior.

Looks like this is the case, yes, it's just very confusing since it stops user from installing unsigned WE extensions while Waterfox and Firefox ESR claim it's supported, i.e. xpinstall.signatures.required, false doesn't help.

I've always had to do this when modifying signed WebExtensions that don't have an ID

Oh so you ran into this issue before? Well I barely have any WE addons since their support in pre-57 versions is quite poor, most of them require 60+ or even 66 anyway, so I never had a reason to modify them.

I don't understand this patch

Previous hotfix forced Firefox to install new certificate into user storage cert8.db/cert9.db, now Mozilla bakes it into Firefox itself.

Petros606 commented 5 years ago

Is it possible the plugins available for download at the Mozilla website have been altered so that only Firefox 66 users can install them?--possibly since Add-on issue started days ago (so-called Armagadd-on-2.0) ?

laniakea64 commented 5 years ago

Is it possible the plugins available for download at the Mozilla website have been altered so that only Firefox 66 users can install them?--possibly since Add-on issue started days ago (so-called Armagadd-on-2.0) ?

I don't think so. The only difference between Firefox 66.0.3 and 66.0.4/66.0.5 is armagadd-on-2.0 fix. And that same fix works for Waterfox to install the addons again.

Petros606 commented 5 years ago

The only difference between Firefox 66.0.3 and 66.0.4/66.0.5 is armagadd-on-2.0 fix. And that same fix works for Waterfox to install the addons again.

Then where's the fix for Waterfox 56?

laniakea64 commented 5 years ago

Merged but not released yet - https://github.com/MrAlex94/Waterfox/pull/940

Ibuprophen commented 5 years ago

I did notice, for the Android App, that regarding some of the Addons I couldn't install, I was able to install only certain older versions/releases of them.

This is an oddity...

~Ibuprophen

LeeBinder commented 5 years ago

one more step and you're there .. ;) (moved up for better accessibility)

Ibuprophen commented 5 years ago

I am rooted...

I've got the Waterfox Beta v56.1.0 (ARM 32-bit) I always install directly from the Play Store.

I took a look and found that it currently has the "cert9.db". Should I try to look within this Github Repo (or the Apk file) for a cert8.db to extract and replace the current one with?

~Ibuprophen

LeeBinder commented 5 years ago

@ibuprophen1 : FIXED!! Here's how:

  1. fired up FX (my fav. Android file manager), tapped onto "System (root)"
  2. navigated to my Waterfox profile (data/data/org.waterfoxproject.waterfox/files/mozilla/abc..xyzDefault): also only cert9.db.
  3. navigated to my Firefox 66.0.5 profile (data/data/org.mozilla.firefox/files/mozilla/abc..xyzDefault): also only cert9.db.
  4. force quit Waterfox, renamed cert9.db -> cert9.db.bak
  5. copied FF profile cert9.db -> WF profile
  6. launched WF, went to Custom UserAgent String, tapped onto "+ Add to Firefox" -> BOOM, INSTALLED!! 🥇 👍 💯

[EDIT]: the same MIGHT apply to Desktop versions, too (copy FF cert9.db {the one updated currently by FF} -> WF cert8.db {the one updated currently by WF}) (don't have any time to test right now)

LeeBinder commented 5 years ago

Just for the record (thanks to Samuel Vuorela for the link) - in case someone wants to compare it w. v.1.0.2 and/ or play with it: hotfix-update-xpi-intermediate@mozilla.com-1.0.3-signed.xpi

Ibuprophen commented 5 years ago

@LeeBinder, I had tried your steps (very carefully too) and didn't resolve the issue for me.

It did make me think a little more about some other directions to try out. I'll definitely let you know of any results on my end.

I did receive the following Popup yesterday before trying your steps out and just remembered about forgetting to provide the screenshot as it is a new one (for me). Also, I was actually in Github when it happened without doing anything addon related too.

New-Error-1

Thanks a bunch! :-)

~Ibuprophen

reallyuniquename commented 5 years ago

You guys shouldn't really tamper directly with cert.db files, just import new certificate as new CA: https://www.velvetbug.com/benb/icfix/icfix.pem (yes, it's the one from hotfix XPI, you can verify it manually)

LeeBinder commented 5 years ago

@reallyuniquename : then please tell ibuprofen how to do that in Waterfox for Android which is lacking any certificate import function in preferences.

LeeBinder commented 5 years ago

@ibuprophen1 :

  1. attached you find the cert9.db from FF Android which works for me in WF Android: cert9.db.zip

  2. paste this into your Android WF address bar:

about:config?filter=xpinstall.signatures.required

and make sure the value is set to false. If it's not, toggle it to true (by simply tapping on it) and try install the add-on again.

  1. which add-on are you trying to install - Custom UserAgent String for testing? If a different one, please post the link, would you.
reallyuniquename commented 5 years ago

@LeeBinder

Waterfox for Android is lacking any certificate import function

Is it though? AFAIK if you open certificate via URL that returns application/x-x509-ca-cert MIME type Waterfox for Android would ask you to install it. I bet you can even install just by visiting file:///sdcard/blabla.crt.

LeeBinder commented 5 years ago

@reallyuniquename : no that doesn't work, just tried: regardless if a) from URL or from file and b) filename extension .pem or .crt, Waterfox simply opens the text file as what it is, as text file.

[EDIT]: here's why:

when you put the certificate on a webserver, make sure it is served with MIME type application/x-x509-ca-cert (or application/x-x509-user-cert for client certificates).

Without this, Firefox will not install the certificate, but download it instead. (source)

Also: Setting up a webserver to automatically serve .crt files as installable certificates for Firefox Android

Obviously Ben from velvetbug was not aware of this.

reallyuniquename commented 5 years ago

@LeeBinder that's weird, this is how certificate installation worked a year ago, although that was vanilla Firefox for Android.

filename extension .pem or .crt

Extension doesn't matter, it's all about MIME type. Try snatching it off the web server that forces proper MIME type (run it on python or something).

Otherwise, yeah, one would need to replace cert.db indeed...

[EDIT]: Well yeah, I didn't imply you should install it from velvetbug site as it sends Content-Type: text/plain; header and I mentioned MIME type thing earlier.

LeeBinder commented 5 years ago

Agree, see my edit above .. ;)

so if someone here with access to a properly configured webserver would upload icfix.pem to their server, best as both .crt and .pem, then test if Waterfox automatically offers to install it when clicking onto the linked file, then we should be all set!

LeeBinder commented 5 years ago

https://mahalo.lima-city.de/icfix.pem ;) @ibuprophen1

Ibuprophen commented 5 years ago

@LeeBinder, okay... Progress...

I had tried your last 2 post suggestions and I can install the Addons now but, their disabled with the "Cannot be Verified" message in the about:Addons screen.

~Ibuprophen

grahamperrin commented 5 years ago

… I'm confident Alex will also release WF 56.x.x with this fix very soon …

In parallel, re: https://www.reddit.com/r/waterfox/comments/bktabg/for_users_of_waterfox_5629_who_may_be_affected_by/emyvez4/ it seems to me that Mozilla's extensions are in the final stages of quality assurance (QA). Interested users can/should track the topic in Mozilla Discourse.

LeeBinder commented 5 years ago

@Ibuprophen1 : step-by-step.. So I guess you cannot ENable them? And have you force-quit WF then restarted?

grahamperrin commented 5 years ago

… force-quit WF then restarted?

From what I found with some prior approaches to importing the certificate:

grahamperrin commented 5 years ago

Nit:

false. If it's not, toggle it to true

Ibuprophen commented 5 years ago

I tried them all... I'm still a determined individual and don't want to lose Waterfox since I've been using it for years on my PC's and Android's.

~Ibuprophen

unicorndreams commented 5 years ago

@LeeBinder - I've just checked, v1.0.3 has exactly the same PEM string as v1.0.2.

LeeBinder commented 5 years ago

@ibuprophen1 : either we or Mozilla (with the user-installable extension for FF 52 through 60 - 👍 @grahamperrin for the link!) will get you back on board until your headaches are gone .. ;)

Are you still using the cert9.db either from me or from your FF? If so, do the following (A):

  1. quit WF
  2. restore your old cert9.db with your root file manager
  3. re-open WF, click onto icfix.pem, import the cert into YOUR cert9.db
  4. quit WF
  5. re-open WF, go to about:addons and check if your add-ons are back

If still not working (B):

  1. quit WF
  2. in Android settings, set your date ahead, e.g. two days
  3. re-open WF, go to about:addons and check if your add-ons are back
  4. set your time/date back to automatic

If still not working (C):

  1. quit WF
  2. backup your cert9.db with your root file manager, e.g. by renaming it so WF will not find it
  3. re-open WF - it should re-create a new virgin cert9.db from scratch. Go to about:addons and check if your add-ons are back
  4. click onto icfix.pem and import the cert
  5. try installing an add-on for testing, like Cookie Quick Manager

@unicorndreams: thanks for comparing v.1.0.2/ 1.0.3 xpi PEM string

MrAlex94 commented 5 years ago

Okay, I've updated the fix pushed by Mozilla: https://github.com/MrAlex94/Waterfox/commit/946ffc1d3d8404f980392f9f353373a7d63506f2

I've tested and seems okay to me, but would appreciate any testing from others as well.

laniakea64 commented 5 years ago

Thanks @MrAlex94 , I tested it on one installation with the previous fix and one without. Seems to work in both cases.