Firefox 42 will require extension signing

[cebamps]

Firefox 42 will no longer allow unsigned extensions, except for the developer and nightly versions. This means that users will not be able to pull the extension and compile it from git anymore.

Since there is no other channel that I am aware of to get an up-to-date Pentadactyl xpi, this means I will no longer have access to the extension when 42 releases using a mainline Firefox release.

This is not yet an issue, but I'm probably not the only person who will be affected, so I think it's worth reporting. Firefox 42 is scheduled to release in November.

https://wiki.mozilla.org/Addons/Extension_Signing https://wiki.mozilla.org/RapidRelease/Calendar https://news.ycombinator.com/item?id=10038999

[behrmann]

Could there maybe be a solution where pentadactyl gets a continuous integration service that could build the xpi, submit it for signing and then put it up for download (like it used to be in the olden days)? As far as I understand the extensions only need to be signed, not hosted on Mozilla's add on platform. Some CI solution could also solve #26

[behrmann]

Just a short an afterthought: This is a relevant problem, as probably no major distribution will disable the signature checking [1] and seeing how many people used to be unable to fix the MaxVersion string in install.rdf, when that came a day or two late, and build pentadactyl from sources I don't see them going through the hassle Mozilla has planned for privately used add ons (making an account, submitting an extension for signing themselves, etc), I think this could be a serious blow to the userbase.

[1] not even Arch Linux will go that direction https://bugs.archlinux.org/task/45900

[behrmann]

Nevermind, although someone has already built some CI [1] an API for signing, that one could hook into is yet non-existant [2]:

Will there be an upload and signing API so I don't have to manually upload each new version of the add-on?

This isn't currently part of the plan for the first version of this project. However, we have received enough requests for this feature that we're looking into ways to make this happen.

So we can only hope that such an API will exist when this is rolled out in FF42, that this is not a priority is quite puzzling, knowing that the validation of add ons is supposed to be automatic [3].

[1] https://github.com/ffledgling/dactyl-build [2] https://wiki.mozilla.org/Addons/Extension_Signing [3] https://github.com/mozilla/amo-validator

[wshanks]

Yes, I have tried to bring this issue up several times over the past five or six months, including opening issue #26 and posting several times on the mailing list and a couple on IRC. I have never seen any word from the developers about this though. The best solution in my opinion would be to actually push releases to AMO when a Firefox update breaks Pentadactyl. If the frequency of updates is too great for full AMO review, the AMO Beta release channel could be used.

[behrmann]

I don't think that is a sensible solution, for one it is rather hostile. Secondly, I do think the devs are very aware of the problem, since @kmaglione works for/with the AMO team and actively on the validation problem, as can be seen from [1]. Thirdly, it is not practically possible, I think to just take over an extension on AMO against the developers wishes und push new releases. This first and foremost is a point for my first one, though, it is hostile and therefore bad.

It is though, a point that @kmaglione and @dkearns are a rare sight in any of the discussions here. It would be tremendously helpful if one could any pointers to where the project is (supposed to be) going and how one cal help.

[1] https://github.com/kmaglione/amo-validator

[wshanks]

Please read my post again. I am aware that @kmaglione works for Mozilla and is one of the primary reviewers of add-ons for AMO. I also used to review add-ons for AMO myself when I had more free time. I didn't say anyone should "take over an extension on AMO." I said the best solution would be to push releases to AMO. Only the developers have access to the AMO account to do this. The review process (which involves a manual review as well as the automatic review from the AMO validator) is time consuming for such a large add-on like Pentadactyl. This is why I suggested the possibility of using the Beta channel (only available to add-ons that have been fully reviewed, which Pentadactyl, as submitted by its developers, has been). I and several other people have offered to help with this in any way possible but the developers have not responded to these offers. See for example:

Issue #26 https://groups.google.com/forum/#!topic/pentadactyl/-XpPCeH_GqY https://groups.google.com/forum/#!topic/pentadactyl/fMr4S-oRKYc https://github.com/ffledgling/dactyl-build/issues/1#issuecomment-76165523

Edit: Thinking more about "taking over" Pentadactyl, one potential option would be for the developers to give their consent to someone posting another listing of Pentadactyl to AMO (with a different listing name to make it clear that the listing was not actually maintained by the developers) and trying to keep it up to date. I don't think anyone would want to do this without developer approval, but if they really aren't interested keeping Pentadactyl updated on AMO maybe they wouldn't mind someone else doing it.

[asmunder]

FYI: I'm the package maintainer for pentadactyl-git on ArchLinux. I've subscribed to this thread and am hoping there will be a workable solution in the end.

Looking more into it, I think the Beta channel proposal sounds good. The Mozilla document even specifies it can be used for nightlies, and the review will be automatic, taking a few seconds. So it seems to me that building nightlies, submitting them to the review, getting them signed and uploading the signed version to the Beta channel could be fully automated. Surely someone should be able to write and host that as a Python/cron script? Heck, I could do it.

[wshanks]

@asmunder Have you looked into automating the extension signing process? My understanding is that there is no API, so it might be necessary to do something like browser automation to upload the xpi via the javascript upload UI.

[asmunder]

@willsALMANJ Yeah, I saw there was no API. I've automated logins, downloads, uploads etc before using urllib2 in Python. If there's actually javascript you need to parse you need to use Selenium instead and it can be more complicated. And sure, you can say it's a form of browser automation, but it's easier to debug, make it work consistently, and then run it as a cron job, as compared to automation plugins for an actual browser.

[wshanks]

Okay, cool. Just making sure you were aware of the complications. I guess the plan for the AUR would be for pentadactyl-git to become pentadactyl-nightly or pentadactyl-beta and pull down a signed version of Pentadactyl from somewhere (addons.mozilla.org or 5digits.org most likely) since pulling from GitHub will no longer work. If there's no word from the developers, you could set up the automated signing process and have it post to the releases section of a GitHub repo as is done here https://github.com/ffledgling/dactyl-build and then have the PKGBUILD pull the XPI from there.

[wshanks]

Even if the developers address the signing issue, a substantial amount of refactoring will be required to keep Pentadactyl functional in upcoming versions of Firefox (or somewhat functional; I'm not sure that it will be possible to recreate all of its functionality once all of the proposed changes are enacted):

https://blog.mozilla.org/addons/2015/08/21/the-future-of-developing-firefox-add-ons

Perhaps the developers' recent apparent lack of interest in promoting Pentadactyl is a sign that they were aware of these changes and aren't planning on trying to keep Pentadactyl working through them?

[barrosfelipe]

@willsALMANJ that should be a new thread. The scope of the problem is much larger.

I'm looking forward to a proper response to the "new news".

[ids1024]

Mozilla really needs an api for this. It is even affecting their own projects:

mozilla/shumway#2329 mozilla/pdf.js#6137

[sigmavirus24]

@asmunder

I've automated logins, downloads, uploads etc before using urllib2 in Python.

Please don't use urllib2. It does not do HTTPS verification and I'd really like to ensure that the uploads are actually secure.

[bittin]

Will you update, the Pentadactyl nightly so it works with the Firefox 42 betas? I wanna try it out :+1: and have to use the latest Firefox Beta as i do the Swedish L10n for Mozilla Firefox and such :D

[polyzen]

@bittin, this issue is specifically for extension signing. You should open another issue.

[jdevelop]

So PD doesn't work with FF 42, confirmed. Any plan to resolve this soon? Thanks!

[behrmann]

That is not a problem of FF though (since signing has been postponed by two versions I think), but the MaxVersion in the install.rdf being set to 41.*

You can change this and rebuild pentadactyl.

[vyp]

@behrmann so firefox 44? source? (also it may be possible to use 'unbranded' versions of firefox regardless)

[wshanks]

@vyp Here is the current plan: https://wiki.mozilla.org/Addons/Extension_Signing

So Pentadactyl will have to be signed to run in the next beta. Release has two more versions before it must be signed.

[vyp]

@willsALMANJ Thanks again.

Looks like you were right, only one commit in the last few months and no developer responses here, future does not look good for pentadactyl.

[scfcode]

Seems sadly grim news for this extension. Especially with no Dev concerns visible yet.

[j127]

"Especially with no Dev concerns visible yet."

It's a bit worrying. Pentadactyl is the main thing that makes web browsing tolerable. The new Firefox user interface is a disaster and Pentadactyl (plus about:config) is the best way around it. There would be a huge Pentadactyl community if it were only released on the official add-on site. The current barrier to entry is too high for most people.

[vyp]

Required extension signing also unfortunately hinders the chances of a maintained fork from coming up.

plus about:config

In fact, it's possible to change these values in the .pentadactylrc itself by using set! commands.

[polyzen]

Fwiw Vimperator is at least maintained, and has signed releases.

@vyp, yeah, it's great :) Helps when maintaining multiple profiles/installations

[j127]

Required extension signing also unfortunately hinders the chances of a maintained fork from coming up.

If the original developers abandon the project, why couldn't a fork be signed under a different name? I thought that the biggest threat to Pentadactyl-like extensions is the new plugin API.

Fwiw Vimperator is at least maintained, and has signed releases.

I tried it, and I don't like it as much.

In fact, it's possible to change these values in the .pentadactylrc itself by using set! commands.

I currently have a user.js file in my profile for that. I'll look into the .pentadactylrc idea though.

[vyp]

Ah I didn't know about user.js, thanks.

why couldn't a fork be signed under a different name?

It can, but that's the point, it has to be done, hence hinders. (because I really don't think requiring extension signing is worth the supposed security benefits, but that's another issue altogether)

[j127]

I really don't think requiring extension signing is worth the supposed security benefits

It's a terrible idea, because it makes Mozilla the gatekeeper for what software can be installed on your computer. That means that if you create a Firefox extension and Mozilla doesn't approve of it for some reason, there is no way to distribute it... (no overrides)

[vyp]

Exactly. But yes, the bigger threat is probably a new deficient plugin api regardless.

[j127]

Yes... I don't think that Mozilla understands why Firefox is losing market share. It's because they don't cater to developers, not because Chrome extensions don't run on Firefox.* What they should do is fund Pentadactyl development and use it to rally developers around how amazing the current FF add-on system is.

• An example: every JavaScript tutorial uses the Chrome dev console as an example. That means every new web developer is going to use Chrome. Developers are thought leaders, so people tend to follow them.

Chrome is actually much slower than Firefox from a users' perspective, because the auto-completion is optimized to send people to Google Search instead of directly to the destination website. Users then click on the disguised Google ads, which gives Google an extra $2-5 (or whatever) for each search done with their browser. Mozilla is trying to compete with Chrome by copying the look and the more limiting APIs, when really what they should be doing is letting users know why Firefox is faster for users and better for developers.

Sorry to go off on a tangent, but I'm a long-time user (since Mozilla Application Suite) who is hoping that the community will rally around better ideas before it's too late.

Maybe some of the Pentadactyl users could put together a community site to teach people about the plugin and let people know why Firefox is good for development? It could be like the old days of SpreadFirefox.com where the community was extremely enthusiastic about getting people to use Firefox. I think that Firefox needs community support like that if it's going to survive at all.

[wshanks]

I know Pentadactyl is a fork of Vimperator, but I have never tried it out. What are the main differences?

I had been planning to try to get a version of Pentadactyl signed as an unlisted add-on for my own use (just change the add-on id to something else and upload it myself) once extension signing is mandatory, but I'm not too optimistic about that. There are already complaints about mission creep with the extension signing program. See the thread created by "Josh" -- maybe that's you -- on the addons.user-experience list where a Zotero developer is complaining about Zotero always getting flagged for manual review and being hassled repeatedly about issues that have already been approved in previous manual reviews. The extension signing program was pitched as preventing malware but is now turning into a detailed code quality review of every extension. Pentadactyl is comparable to Zotero in terms of size and the amount of Firefox internals it touches, so it would likely also have an arduous review process for each update.

All of that is to say that it might be easier to contribute patches to Vimperator to bring it closer to Pentadactyl than to try to maintain a working, signed version of Pentadactyl since Vimperator already has at least one developer willing to help it get through the signing process. Strong developer support will also be needed to deal with the other upcoming changes to the Firefox extension API. I haven't tested it myself, but I have seen others say that Pentadactyl does not work well with Electrolysis which will become the default at some point. Also, Pentadactyl currently uses a lot of functionality not available in the Chrome WebExtensions API, so it will not be possible to maintain it once XUL is phased out unless there is a strong developer voice advocating for Mozilla to implement the missing functionality into Firefox's WebExtensions.

[mariusor]

With the risk of propagating the things I'm complaining about:

How about you guys take these discussions that are not related to the issue at hand to somewhere else?

[wshanks]

Hmm, I'd say all of the posts (except yours) are related to the issue at hand, though they do not concretely move towards a resolution of the issue. I'll open a new issue any way. However, if you're expecting more than discussion about extension signing out of this issue, you're likely to be disappointed.

[j127]

See the thread created by "Josh" -- maybe that's you

That's me. I'll write more over in issue #99.

[polyzen]

It appears that there is now an API for extension signing: https://olympia.readthedocs.org/en/latest/topics/api/signing.html. Hopefully someone can look into this.

https://github.com/mozilla/pdf.js/issues/6137#issuecomment-156743676 by @timvandermeij

[bmcorser]

So is there a way to hack Pentadactyl to work with FF42? I'd like to keep updating my version of FF (even after 42) so would be useful to know how I can make Pentadactyl follow along.

[asmunder]

Buried somewhere in the comments above is the fact that mandatory signing has been postponed to FF 44. You need to manually increase the MaxVersion string in the install.rdf file though. On 30 Nov 2015 07:16, "B M Corser" notifications@github.com wrote:

So is there a way to hack Pentadactyl to work with FF42? I'd like to keep updating my version of FF (even after 42) so would be useful to know how I can make Pentadactyl follow along.

— Reply to this email directly or view it on GitHub https://github.com/5digits/dactyl/issues/79#issuecomment-160528320.

[wshanks]

Also, for Firefox 43 (released later this month) you will have to disable xpinstall.signatures.required in about:config (this option will be removed in FF 44 as @asmunder indicated).

There was some good news announced today regarding getting Pentadactyl signed for use in FF 44 and beyond. There is now a signing API and it will be fully automated soon. With the previous (and current for the time being) system, Pentadactyl would have triggered a manual review making it difficult to get it signed. With this change, even if the developers don't feel like signing Pentadactyl themselves, we now have the option of changing the Pentadactyl's addon id (since it must be unique and pentadactyl@dactyl.googlecode.com already is taken on addons.mozilla.com) and submitting it for signing ourselves, perhaps as part of automated build process like https://github.com/ffledgling/dactyl-build

[lf-]

Also: if you're using iceweasel and the package is installed into /usr/share/firefox, it will work because of a debian patch to stop requiring signing of addons in there.

[wshanks]

Since Firefox 44 is coming out soon, I put together some scripts to sign Pentadactyl using Mozilla's API and upload the signed xpi file to GitHub for distribution. The xpi files are at https://github.com/willsALMANJ/pentadactyl-signed/releases. The scripts are also in the repo if you want to use them to sign to set up automated signing/updating of Pentadactyl for yourself rather than trust my xpi's (you'd have to edit the files to change the addon id to something else unique and change the GitHub repo url to your own).

[polyzen]

Looks like the title needs an update since we're well past 42 :p

[polyzen]

@willsALMANJ, does that run automatically on every commit/night or manually?

[wshanks]

I run it manually when there are new commits (I subscribe to the feed). I wrote a systemd timer to run it nightly but it's not running and I haven't had a chance to debug it.