5digits / dactyl

Pentadactyl and other related Gecko extensions
Other
467 stars 98 forks source link

Future development of Pentadactyl #99

Open wshanks opened 8 years ago

wshanks commented 8 years ago

The commit logs have been fairly quiet recently (1 commit over the last 3+ months) and the developers have not responded to several mailing list posts and GitHub issues posted over the last seven months or so regarding upcoming Firefox changes that will render Pentadactyl unusable. So the question is where does Pentadactyl go from here. Here are some major roadblocks for Pentadactyl that need to be addressed:

  1. Distribution / promotion

    The version of Pentadactyl hosted on AMO has been incompatible with Firefox Release for most of the last two years. The Pentadactyl website has been out of date for years as well. Now, even the nightly builds are often unreliable or out of date. An updated version of Pentadactyl needs to be the first version users find when they search AMO or the web. Issue #26 has more details about this issue.

  2. Extension signing

    On January 26, 2016, Firefox Release will require that extensions be signed, rendering Pentadactyl unusable unless it is signed. To be signed, it must be uploaded to Mozilla's site by the developers. Alternatively, it could be forked and given a new add-on id and that version could be uploaded by a third party. However, Pentadactyl is very complex and will not pass the automated tests part of the extension signing process. Getting something as complex as Pentadactyl through a manual review will be difficult and require a lot of effort (developers who understand the Pentadactyl code base will have to advocate for why certain sections of the code are secure and should be allowed through Mozilla's approval process despite tripping automated test failures). Issue #79 has more on this subject.

  3. Electrolysis

    Electrolysis (multi-threaded version of Firefox) will at some point become the default and then later on the shims that allow add-ons written to work with single-threaded Firefox to work in Electrolysis will be removed. More testing is needed but as is Pentadactyl will be unusable with Electrolysis and without the compatibility shims. This issue was first raised in issue #50 (edit: originally this said #84), though there is not much content there right now.

  4. Deprecation of XUL and XPCOM

    Further into the future (~18-24 months), Firefox plans to transition away from XUL-based add-ons. At that point, only add-ons using the Add-ons SDK or the WebExtensions API will work with Firefox. All signs point to the Add-ons SDK also being phased out slightly further into the future. The WebExtensions API does not allow for all of the functionality of Pentadactyl (this is why there is full-featured equivalent of Pentadactyl for Google Chrome and why the closest thing, Vimium, has to use some convoluted code to get as much functionality as it can). Developers who understand Pentadactyl and the WebExtensions API well need to start advocating now for new API's that will allow Pentadactyl's features to be created with WebExtensions.

Some avenues that could be explored:

polyzen commented 8 years ago

Vimperator will also have to be rewritten. It would be something if these two could merge in that process..

vyp commented 8 years ago

I'm not actually sure the vimperator developers would be interested in pentadactyl-like features. As I understand it, there was a little 'bad blood' between both parties at the time of the fork. On the other hand, that was quite some time ago, so maybe I am not recalling correctly, or maybe the circumstances have changed.

j127 commented 8 years ago

I'd be interested in helping to save Pentadactyl.

polyzen commented 8 years ago

@vyp, perhaps Vimperator will have to be forked again after the rewrite :p

vyp commented 8 years ago

https://superuser.com/a/261731 may also provide a little more context regarding vimperator vs pentadactyl (note that that's the main developer of vimperator at the time).

j127 commented 8 years ago

I've tried it and switched back to Pentadactyl.

barrosfelipe commented 8 years ago

Why give no answer to the community? "Hey guys, we are not going to keep working on this. Thanks for everything." or "Hey, we are alive and trying to sort things out."? It's not like they don't see the Github notifications. It's not like @kmaglione is unaware that the community expects an answer from him. Step up guys. I know this is "free" stuff, so just give us a "get out" or "we got this". People like me came to depend on Pentadactyl to do our everyday computer stuff.

oddRaven commented 8 years ago

I started using Pentadactyl because Vimperator wasn't able to do something, switched back recently due to incompatibility with Firefox 42. It works, but it doesn't feel as advanced as Pentadactyl, it doesn't try to do everything in the commandline. I'd like to help keeping Pentadactyl stay alive but I don't have any experience working on Firefox extensions (yet).

amosbird commented 8 years ago

Going back to Vimperator is not an option for me.

congma commented 8 years ago

I wonder how this will affect Pentadactyl: Firefox Bug 1222546 - Product plan: remove support for heavyweight themes.

Does Pentadactyl use themes?

polyzen commented 8 years ago

@congma, I doubt that will affect us. You can change the colorscheme of penta itself, but that doesn't affect ff's theme

wshanks commented 8 years ago

@congma The previous announcements already called out major changes to Firefox that require major refactoring of Pentadactyl, so this announcement doesn't have too much direct impact (Pentadactyl is an extension, not a theme). However, the changes to themes do indicate a restructuring of how Firefox allows its UI to be modified, so that might impact how the statusbar and command line would have to be implemented.

The discussion in that bug is very relevant to the issues facing Pentadactyl though because themes are undergoing similar changes to extensions to align with the changes to Firefox/XUL/XBL and because @kmaglione (Pentadactyl developer) is one of the main participants. He even references Pentadactyl at the end of this comment (the only time I have seen him acknowledge the impact of the upcoming changes on Pentadactyl:

I do think that it sucks that there's going to be such a sharp transition for add-on developers, and I say this as the developer of one of the largest, most complex extensions for Firefox power users. But I also think that it's necessary, and in the end will make things better for just about everyone.

  • Kris Maglione
polyzen commented 8 years ago

@willsALMANJ, and just two days ago. Good to hear

xificurC commented 8 years ago

I'd love to hear some update from the devs too. I use pentadactyl everyday and would like to see it getting a bit more active, while there is still time. vimfx and vimperator are not even close to what pentadactyl provides.

ghost commented 8 years ago

@willsALMANJ I just stick with @ffledging's dactyl releases repo https://github.com/ffledgling/dactyl-build/releases

Another option would be to use firefox ESR version in the meanwhile.

If penta really dies I would just give qutebrowser a try.

vyp commented 8 years ago

@k0377 qutebrowser is pretty great, I've been using it for a short while now as I look to find alternatives to pentadactyl. However there's still a lot of things where qutebrowser doesn't even come close to pentadactyl. But hopefully that will be rectified in the not so distant future. Another project people here might want to keep an eye on is https://github.com/meh/miserve.

j127 commented 8 years ago

If penta really dies...

That is a frightening thought, since the default FF interface has been going downhill rapidly and Pentadactyl is what makes Firefox bearable. The alternatives are not suitable, and who wants to use overly simplified and restricted tools that exclusively cater to the lowest common denominator. (I'm talking about the trend towards over-simplification of UIs and enforcing one way to do things, with no overrides for power users. That way of thinking leads to massive hype over inefficient crap like touch screens and tablets/phones that have no root access.)

What if there were a movement to build a pro-Firefox developer community around Pentadactyl? Many developers would rally around an amazing tool like this. Mozilla should not only ensure that Pentadactyl doesn't disappear via limited APIs, but they should actively fund and promote its development. It's the add-on that makes Vim-using programmers say: "My mind just got expanded, and I'm never leaving Firefox."

Developers are a major force behind browser adoption. Non-tech users get their opinions from tech-savvy users. Chrome actively caters to programmers. Virtually every JavaScript tutorial on the Web uses Chrome dev tools. Mozilla is critically lacking in effective developer outreach. Pentadactyl could be a killer marketing tool for Mozilla. If Mozilla wants programmers to build stuff for Firefox and Firefox OS, then this is the kind of stuff that will draw them in. Limiting the API and restricting which extensions can be installed is going to have the opposite effect.

j4hangir commented 8 years ago

@j127

I suppose the only reason many of us here keep using Firefox is Pentadactyl, and not much else.

Considering these rather insurmountable issues that Mozilla is imposing, given that I know very little about the add-on submission process however from the sound of it seems extremely tedious and difficult, especially when it comes down to such complex add-on like Pentadactyl, I as well agree that in order to ensure its continuance, the best course of action is most probably to rally around Pentadactyl and make it an indispensable, rule-break worthy extension.

Let Mozilla spend some of their resources on something that is truly worth it.

vyp commented 8 years ago

@j127 @j4hangir it's probably relevant to link this thread which gathered a lot of attention (relatively).

wshanks commented 8 years ago

It may be necessary to port/rebuild Pentadactyl piece by piece to work with future version of Firefox. For help with prioritization, I'd be curious to hear which features keep people using Pentadactyl over Vimperator (assuming it is easier to port features common to both since Vimperator devs will also be working on those).

noctuid commented 8 years ago

@willsALMANJ I have a list here with the main thing being pentadactyl's group feature. I may be incorrect about some points (or missing some) since I haven't really used vimperator extensively.

xrchz commented 8 years ago

switched to vimperator while {#105, #95, #93, #104} etc. are still open for pentadactyl. very interested in pushing for one or both of these extensions to have a well-supported future.

j127 commented 8 years ago

The lack of response here makes me worried that the plan is to let Pentadactyl die off when the APIs are changed. If no update is posted here soon, maybe it could be assumed that the project is in the process of being abandoned?

A simple: "yes, we're going to make sure that Pentadactyl still exists in 2017" or "sorry guys, we aren't going to maintain it after 2016" would settle the worries of hundreds of people. :)

I think that, one way or another, a group of power-users should organize to make sure that these add-ons still exist in a couple of years, while also promoting them as one of the main attractions of Firefox as a browser. If programmers like Firefox, its market share will increase. If programmers switch to Chrome (because nothing better than Vimium might exist on Firefox if the APIs become too limited), then it could have a huge negative effect on Firefox's adoption be deflating the enthusiasm of Firefox's core supporters.

I like Firefox, because it is customizable in ways that other browsers aren't. If that changes, it will become just another restricted, appified piece of crap that is aimed at the lowest common denominator.

vyp commented 8 years ago

If no update is posted here soon, maybe it could be assumed that the project is in the process of being abandoned?

Yes I agree, it's what I have been thinking myself. The developers are definitely seeing these dialogs, just not responding, so in a practical sense it is already abandoned. Of course, I don't know any of the developers myself, maybe there are legimitate real-world issues that prevent them from tending to pentadactyl at the moment. Maybe they have plans to respond here, or maybe even have plans to continue future development of pentadactyl. But without any hint or sign of the current status, I think we have to assume the worst. (The only thing we have of recent is that @dkearns merged #103 just under a week ago. edit: also the quote by @kmaglione that @willsALMANJ posted above)

elmindreda commented 8 years ago

Please give them the benefit of the doubt. A thread like this can be pretty intimidating to respond to even without people assuming things.

wshanks commented 8 years ago

It is a frustrating situation (especially since a two sentence message posted any time in the last six to nine months would save lots of people from hand-wringing over how to handle the upcoming Firefox changes), but I do try to give the developers the benefit of the doubt since I am so grateful for the work they have done.

My interpretation of the developers' actions is that some time over the last two years they stopped treating Pentadactyl as a public software project and started treating it as a personal project that they happen to share with the public (maybe that's all they have time for right now). That's how I interpret the fact that even now there are occasional new commits, though the web site is very much out of date, as is the addons.mozilla.org listing, the nightly builds have not worked consistently for many months, and the developers rarely appear on the mailing list or issue tracker. I think the current trajectory for Pentadactyl is that it will likely become unusable some time during the Electrolysis transition (could possibly be default in Release Firefox by spring 2016 though I don't know the plan for allowing it to be turned off) with the developers possiblity posting a few compatibility commits before then for other issues. I'd love to be proven wrong though.

My hope with this issue was to provide a place where people with the skills and resources (time) to keep Pentadactyl alive could come forward to coordinate. It is one of the most complicated Firefox addons ever written though, so I would not be surprised if those people do not exist (I have been trying to dig into #95 in my free time but haven't gotten close to figuring out what is going on). I would like to think any fork/rewrite would allow the developers to have whatever involvement level they would want to have.

prikhi commented 8 years ago

I love pentadactyl and am interested in helping keep it alive. I don't have the time to coordinate this, but I can help write code, provide a build server, etc.

mr337 commented 8 years ago

Same here with @prikhi, use pentadactyl everyday and be a shame to see it die off.

facaiy commented 8 years ago

Pentadactyl is one of the reasons why I stay at Firefox. I can't image how life goes on without it.

PLumowina commented 8 years ago

Pentadactyl is first plug-in I install on every damn computer which I use, Lets do something that it will not disappear, unfortunately my knowledge about programming is very poor (I write some simple bash scripts - thats all)

josch commented 8 years ago

Since the future of pentadactyl looks bleak and there seem to be other people looking for alternatives, I wanted to put the result of my search into this bug:

There seems to be an abundance of projects that try to do a browser from scratch but I think the major downside of that approach is that existing extensions will not work with them. So maybe a good bet would be to decide for a browser supporting XUL extensions like conkeror or pale moon.

On the other hand with firefox deprecating XUL, it might happen that important addons like addblockplus or noscript will soon not offer their extensions as XUL anymore...

It is not clear to me which of the above projects managed to collect enough support behind them to actually be a more long-term viable option for me to also invest my own time in.

The-Compiler commented 8 years ago

I want to add my own project, qutebrowser - I started it as a dwb replacement 2 years ago as dwb was getting less and less love (and is now officially unmaintained, and unusable due to an instant segfault on Archlinux).

In the somewhat-near future, I plan to port it to QtWebEngine (so it's essentially Chromium as a backend), add a Python plugin API, and hopefully at some point add some degree of support for WebExtensions.

I also listed some other projects I know of at the bottom of that page. From my point of view, it looks like qutebrowser, vimb and uzbl are the only non-addon projects which are still really active.

Also note uzbl has had its last commit 6 days ago (in the next branch), it just hasn't seen a release since 2012.

Superpat commented 8 years ago

Well pentadactyl has been officially disabled by my firefox, It really is a shame. I'll keep an eye on the repo, but I'll try out uzbl and surf, see if they work out.

iegorka commented 8 years ago

Like temporary solution for FF43 you can set xpinstall.signatures.required => false (in about:config) and pentadactyl continue to work well.

Superpat commented 8 years ago

Oh really? I'll try it out promptly.

Edit: Doesnt seem to be doing anything, I deactivated everything xpinstall related and the message that dactyl is incompatible does not change.

j127 commented 8 years ago

Like temporary solution for FF43 you can set xpinstall.signatures.required => false (in about:config) and pentadactyl continue to work well.

How long is that going to work? I think it's at the emergency stage now.

It's really tragic to see Mozilla destroy what could be one of of their biggest developer rallying points around Firefox. Instead of building community around it (whether Pentadactyl or Vimperator), they're doing things that will cause it to die off.

prikhi commented 8 years ago

I wrote a script to pull the git repo, modify the name, id, version, maxVersion in install.rdf, make the XPI and upload it to the AMO for hosting & signing.

You can install it from here: https://addons.mozilla.org/en-US/firefox/addon/pentadactyl-nightly-unofficial/

I'll have it run as a daily cronjob.

prikhi commented 8 years ago

URL changed to https://addons.mozilla.org/en-US/firefox/addon/pentadactyl-nightly-unofficial/

If you don't trust my builds, here's the script so you can run it yourself:

#!/usr/bin/env bash
# This script clones the latest pentadactyl from github, modifies the max
# version and uploads it to the AMO for signing.
#
# Requires: git, make, pyjwt, python3, sed, zip
# By Pavan Rikhi<pavan.rikhi@gmail.com> on 2015-12-17
BUILD_DIR="${HOME}/dactyl-nightly"
GH_USER=""
GH_PASS=""
DACTYL_REPO="https://${GH_USER}:${GH_PASS}@github.com/5digits/dactyl.git"

AMO_API_KEY=""
AMO_API_SECRET=""
ADDON_ID=""
ADDON_NAME="Pentadactyl Nightly - Unofficial"
ADDON_VERSION="$(date +%Y.%m.%d.%H.%M.%S)"
ADDON_MAX_VERSION='45.*'

main() {
    git clone "${DACTYL_REPO}" "${BUILD_DIR}"
    cd "${BUILD_DIR}/pentadactyl/"

    # Modify Add-on Details
    sed -i "s|em:name=.*|em:name=\"${ADDON_NAME}\"|" install.rdf
    sed -i "s|em:version=.*|em:version=\"${ADDON_VERSION}\"|" install.rdf
    sed -i "s|em:id=\"penta.*|em:id=\"${ADDON_ID}\"|" install.rdf
    sed -i "s|em:maxVersion=.*|em:maxVersion=\"${ADDON_MAX_VERSION}\"/>|" install.rdf

    # Make the Add-on
    make xpi
    XPI_FILE="pentadactyl-${ADDON_VERSION}.xpi"
    XPI_PATH="../downloads/${XPI_FILE}"

    # Generate a JWT Token
    TOKEN_TIMESTAMP=$(date -u +%s)
    JWT_TOKEN=$(python <<PYTHON
import jwt, json, random
data = {
    'iss': "${AMO_API_KEY}",
    'jti': str(random.random()),
    'iat': ${TOKEN_TIMESTAMP},
    'exp': ${TOKEN_TIMESTAMP} + 59,
}
print(jwt.encode(data, "${AMO_API_SECRET}", algorithm='HS256').decode())
PYTHON
    )

    # Upload it to the AMO
    curl "https://addons.mozilla.org/api/v3/addons/${ADDON_ID}/versions/${ADDON_VERSION}/"  \
        -g -XPUT --form "upload=@${XPI_PATH}"   \
        -H "Authorization: JWT ${JWT_TOKEN}"

    cd "${HOME}"
    rm -rf "${BUILD_DIR}"
}

main
vyp commented 8 years ago

How long is that going to work?

I suppose it will work for firefox 43. But firefox 44 will be released january 26 2016, with which you cannot change xpinstall.signatures.required (unless they decide to postpone it again). I think you can still use an 'unbranded' build though. Of course, that by itself is not sustainable, pentadactyl still has to be maintained, otherwise the deprecation of XUL will the end of pentadactyl as decribed in the OP.

prikhi commented 8 years ago

At some point either someone else will take over this, the original developers will return, or I will fork the project in order to save it. I've already forked and saved one abandoned FF add-on(pencil) and that one will require a huge port from XUL to WebExt as well... So my time will be limited but at the very least I'd be able to bump the maxVersion, merge PRs and release it on AMO...

Ideally, I would be contacted by the authors or I would contact the authors(@dkearns, @kmaglione) about taking over or becoming a collaborator on GitHub & AMO. But if I can't reach them I guess I'd have to fork the project instead(decidactyl anyone?)...

mr337 commented 8 years ago

@prikhi If you end up becoming a maintainer let me know if you need any help. Not an expert of FF plugins directly, but anything else I can prob help with.

wshanks commented 8 years ago

@prikhi Thanks for setting that up. Someone needed to do it.... I don't know if @dkearns plans on pushing a fix for #108 like he has done for previous issues but that one looks pretty serious, so we should probably look into it soon and be prepared to rebase your script on a fork of the dactyl repo if @dkearns doesn't merge the fix since Firefox 44 is only about a month away.

I'd still like to see #95 tracked down as well....

prikhi commented 8 years ago

@willsALMANJ Thanks for pointing me to those issues, they should be a good test to see if I'll be able to comprehend the code. I don't see any developer docs so it might be a bit of a struggle...

Also Mozilla is taking their time with the add-on reviews. The unofficial nightly is still in the same spot after 2 days and my other plugin has only moved forward one spot after 3 days. The nightly build is number 153 in the queue... So it might get a full review & be signed in like.... a year and a half...

So now I'm thinking I'll keep that in the queue and also release nightly builds that are unlisted on addons.mozilla.org so that they will be signed immediately. Maybe within the week, but definitely before FFv44.

wshanks commented 8 years ago

Well, @kmaglione seems to have fixed those two issues with bc9eb79fb310dac1ea8dd304bac4db996c51e6d8 or 65725c95164c6da1fe5bedbeafa651c06d08b72e. You can still look at those commits to see the kind of work involved in maintaining Pentadactyl. It's one of the biggest Firefox addons and Firefox is a constantly shifting base underneath it....

Because of its size, I would not expect a quick review of Pentadactyl from Mozilla -- multiple months would not surprise me. @kmaglione is one of the reviewers so maybe you will get an interesting response from him.

Since the devs do seem to be posting patches still, it is probably best to focus on keeping a compiled and signed version of Pentadactyl available for download, since I haven't seen any signs from the devs that they are interested in doing that part (last nightly is still from August). Getting it reviewed on AMO would be best because AMO takes care of signing, hosting, and automatic updates, but it might be necessary to have it signed as an unlisted addon and do the hosting elsewhere possibly on GitHub like @ffledgling does at https://github.com/ffledgling/dactyl-build/releases.

One promising sign is that with 65725c95164c6da1fe5bedbeafa651c06d08b72e @kmaglione hinted at possibly getting Pentadactyl to work with e10s which would be awesome. That would ensure that Pentadactyl could survive at least as long as XUL does.

j4hangir commented 8 years ago

Nightlies hosted at http://5digits.org/nightlies seem to be recent and do work in FF 43.0.2, granted the setting signatures.required => false was set.

There aren't many bugs as far as I can tell, besides a few mapping issues and broken tags (for the Nth time).

insidewhy commented 8 years ago

Does this remind anyone of a friend who goes out on a date with someone, then texts them several times over the next two weeks trying to arrange another date without hearing anything back.

The maintainers can't even be bothered to reply to a single one of your concerns, this relationship... is already over. I don't like vimperator so much, but vimperator is like, the not so hot date, the one that actually responds to you. And you're lonely... so you keep dating that person.

insidewhy commented 8 years ago

@vyp The bad blood you speak of is actually quite hilarious. The vimperator developer was soliciting donations for vimperator despite not having committed a line of code in months. At this stage a bunch of the developers who were actively worked on it called him out on this, and he basically ignored them. So then they forked off pentadactyl. Then at some point a few months later, the vimperator lead starts writing sarcastic/rude/passive-aggressive stuff about pentadactyl in various places (including the vimperator website itself).

Last I saw about this was an e-mail from some pentadactyl dev to the vimperator dude saying something like "Hey man, don't be a hater."

And the story is over.

polyzen commented 8 years ago

Mr Friend, there's been at least one developer still committing code here.

On Tue, Jan 5, 2016 at 1:30 PM, Mr Friend notifications@github.com wrote:

@vyp https://github.com/vyp The bad blood you speak of is actually quite hilarious. The vimperator developer was soliciting donations for vimperator despite not having committed a line of code in months. At this stage a bunch of the developers who were actively worked on it called him out on this, and he basically ignored them. So then they forked off pentadactyl. Then at some point a few months later, the vimperator lead starts writing sarcastic/rude/passive-aggressive stuff about pentadactyl in various places (including the vimperator website itself).

Last I saw about this was an e-mail from some pentadactyl dev to the vimperator dude saying something like "Hey man, don't be a hater."

And the story is over.

— Reply to this email directly or view it on GitHub https://github.com/5digits/dactyl/issues/99#issuecomment-169089960.

j4hangir commented 8 years ago

Yet it is worrisome that none had bothered to write a single comment here. I am not that worried though, I've used Palemoon before and it was a decent fork of Firefox, pretty performant as well. I suppose that will keep the plugin alive for hopefully a few more years.

wshanks commented 8 years ago

I think the best description of Pentadactyl's current state is that is a private Firefox add-on that @kmaglione and @dkearns write for themselves (they both are still contributing commits) and happen to post the code publicly. I'm very grateful to them for doing so because the latest commit still works well with the latest version of Firefox.

I'm not sure where that leaves the questions raised in this issue. As long as they continue to post their commits publicly, it is most efficient to just use their code since they know it best and can best update the code to handle breaking changes to Firefox. On the other hand, we have no communication from either of them and they don't make much effort to make Pentadactyl discoverable/usable by others, so they could just cut off updates at any time for all we know.