Port to WebExtension - Githubissues

SammysHP commented 7 years ago

Sören Hentzschel has published a roadmap regarding multiprocess architecture and addon compatibility: https://www.soeren-hentzschel.at/firefox/so-geht-es-weiter-mit-dem-multiprozess-firefox-2/

Especially two releases are important for Vimperator:

Firefox 53 (2017-04-18): Only WebExtensions allowed on AMO. This means that new developers cannot create new unlisted addons for signing.

Firefox 57 (2017-11-28): Only WebExtensions allowed in Firefox and AMO. Vimperator will not work with this release anymore.

Please note that the developer version of Firefox might drop support for XUL- and SDK-addons already some months earlier, maybe in June!

I have a really bad feeling about this. Considering the development activity during the last months I don't think that we will have ported Vimperator to a WebExtension in time. This process requires almost a complete rewrite of Vimperator and a lot of features aren't possible with WebExtensions at all. Don't get me wrong, but I fear that this is the death of Vimperator. :(

See also #211 for some more details, which focusses on e10s compatibility in general.

Chatroom

I created a chatroom on Gitter where we can organize and talk about the next steps: https://gitter.im/vim-webextension/Talk

hejops commented 7 years ago

If all else fails, could focus perhaps be shifted to compatibility with Pale Moon?

timss commented 7 years ago

Also relevant: https://bugzilla.mozilla.org/show_bug.cgi?id=1215061#c22

I have a really bad feeling about this. Considering the development activity during the last months I don't think that we will have ported Vimperator to a WebExtension in time. This process requires almost a complete rewrite of Vimperator and a lot of features aren't possible with WebExtensions at all. Don't get me wrong, but I fear that this is the death of Vimperator. :(

It would indeed seem that the best possible and realistic outcome now would be to lag behind on Firefox releases for some time until a rewrite is completed, or even started.

At some point it stops making sense to even maintain the current code base. I would hope that we could get a working release (especially on AMO) for the next ESR (FF52), which in my opinion would be a good option to stay on for some time as releases after 52 will be increasingly hard to maintain.

Regarding the option of Pale Moon, there's a pretty interesting discussion on that in this Pentadactyl issue. I don't personally think it's a good idea.

https://github.com/5digits/dactyl/issues/99

svarogg commented 7 years ago

I would really love to help with the effort. However, a few pointers on where can one start would be very helpful.

I've done the reading, and as I understand we are facing two major roadblocks:

New APIs means a complete or at least a very extensive rewrite.
Some APIs are not available, which makes some features hard or impossible to implement.

A couple of thoughts: I think point No 1. warrants a new repository or at least a clean branch, especially if we are talking about a complete rewrite.

To address point No. 2 it would be nice to have a list of missing APIs, and might start thinking about workarounds.

I have much development experience, but none in browser extensions and haven't touched JS for a couple of years. So I don't think I am fit to lead this effort, but I would be more than happy to follow if pointed at the right direction.

mavaa commented 7 years ago

Hey, I just started using Firefox and vimperator after being a long-time Chromium/Vimium-user (Google Amp on my phone slowly made me turn away from google). I love the experience of using gui=none so far and would love to contribute some with keeping this project alive!

That being said, I don't have any experience writing Firefox plugins besides building the latest xpi a few days ago, though I'm a pretty decent developer working mostly in web. I'm guessing I should start by reading a bit about writing plugins in general, would reading up on WebExtensions be the right place to start right now? And/or should I take a look at some open issues already around?

Nefari0uss commented 7 years ago

@marza91 If I had to guess, I'd say Web Extensions might be worth looking into. Someone is gonna have to start a port and the earlier the better is my likely guess.

mavaa commented 7 years ago

This is probably possible to find out somewhere, but is Vimperator a "XUL/XPCOM" or "Add-ons SDK" extension?

schemar commented 7 years ago

Count me in when moving to WebExtensions 👍

SammysHP commented 7 years ago

@svarogg A new repository is a good idea. Yes, it would be a complete rewrite. With a new repository it would be easier to manage issues etc.

For the start we need some experienced software engineers to create an initial framework. Later individual developers can contribute additional features individually. There will be a long design phase where we collect requirements, develop a solid foundation for the addon and where we can also analyze which features are missing in the webextension standard.

@marza91 Vimperator is a XUL addon.

SammysHP commented 7 years ago

Oh, I forgot something: The new version would be something completely new. There will be many things that will be different from what we are used to. It would be fantastic to forget the reasons that led to the fork and work together with the people from pentadactyl. They have the same problem as we have. And most developers joined after the fork. It makes no sense to create two new addon with the same functionality.

cornop commented 7 years ago

Personally, I will be switching to VimFx. It doesn't change the UI, which is one thing I liked about Vimperator, but that won't be possible in the future anyway. It seems to be ahead of Vimperator in terms of Firefox's changes (it already supports multiprocess), is actively developed and has a roadmap for WebExtension support.

I'll just throw this in the room: Maybe the remaining developer power should be moved to this project.

Dioxylin commented 7 years ago

Regarding Pale Moon, their roadmap, at https://www.palemoon.org/roadmap.shtml, explicitly states that they are keeping XPCOM and XUL extensions support, unlike Firefox.

mavaa commented 7 years ago

The ability to change the GUI and the way it completely changes how you use the browser is what I like most about Vimperator as well, so VimFx seems like a downgrade to me. Not to talk down about the Add-on itself, I bet it works great, it's just not exactly what I want (I tried it briefly before changing to Vimperator).

Would it be an idea to start by assembling a feature-list and find out exactly what can and can't be ported? I'm also lending a bit towards switching to Pale Moon if there's interest in keeping the XUL version going and most features can't be ported.

gkatsev commented 7 years ago

Collecting a feature list and categorizing it into "supported by webext" sounds like an awesome idea.

xdml commented 7 years ago

I follow your discussion about moving forward to WebExtensions age. I do not know exactly what is possible and what not, but from the discussion I got feeling, that interface rewrite by XUL will not be possible.

My question is, can be some of these issues solved by theme? So basically some XUL features will be done by theme?

tfrangio commented 7 years ago

I would like to throw my hat in the ring as a developer that can help with the port. Do not have time to lead the project but as soon as we find a leader(s) I can take on tasks.

On Fri, Feb 10, 2017 at 3:22 AM, xdml notifications@github.com wrote:

I follow your discussion about moving forward to WebExtensions age. I do not know exactly what is possible and what not, but from the discussion I got feeling, that interface rewrite by XUL will not be possible.

My question is, can be some of these issues solved by theme? So basically some XUL features will be done by theme?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vimperator/vimperator-labs/issues/705#issuecomment-278884651, or mute the thread https://github.com/notifications/unsubscribe-auth/AAdUdoth77xv24B8GON7AAxRfimW-gS6ks5rbB4ygaJpZM4L6u97 .

timss commented 7 years ago

New repo would be practical due to the high number of unresolved issues in this repo, but it would also mean that one can't as easily refer to still active issues using #705. There's also lots of commit statistics, links to the repo on the Internet etc. As a minimum, the main repo would have to include information and links to the other one.

That being said, I don't plan on manage the move to a new repo and/or branch, so it'll be up to whoever makes the call. Just thought I'd point out some possible pros/cons.

lilydjwg commented 7 years ago

You can easily refer to issues in other repos by pasting the full links, or use user/repo#xxx syntax.

SammysHP commented 7 years ago

Yes, cross-linking issues is easy. The new code would not have much in common with the old Vimperator code, so I don't see a problem. As we will start from scratch, most issues would have no relevance anymore.

timss commented 7 years ago

I'm well aware, @lilydjwg. My point is that there's some history that comes with all the issue numbers. In the event a new repo is created, an issue discussed with number e.g. 77 could be different, even though an issue 77 in the old repo could have historical value. Not a big problem, but it's not just about not having to write user/repo#N vs #N.

fakedrake commented 7 years ago

Because there was some talk about palemoon and other browsers. I switched temporarily to IceCat because vimperator still works properly there.

It is extremely unfortunate tha palemoon does not run on OSX.

SammysHP commented 7 years ago

@timss I don't understand what you mean. The new WebExtension version of Vimperator would be a completely new program. There are no bugs, only feature requests (many more than there are now because it won't have any features at the beginning). So in the rare situation that it is necessary to reference an old issue it would be no problem to use the repo#N syntax.

fakedrake commented 7 years ago

It wasn't clear from my comment but I would agree with@SammysHP as the current version of vimperator should still work with current/legacy browsers like icecat and palemoon.

EDIT: also I believe the (potential) shortcomings of github's issue tracker should probably not be taken into account in this decision.

timss commented 7 years ago

Just thought I'd give my two cents, issue linking wasn't in itself a huge motivation. Please, don't let my remark become too much of a focus, I don't feel too strongly about it :)

jrmiller82 commented 7 years ago

So.... firefox is doing something internally that's going to break vimperator? (Or already broke it? I'm on FF 51 and lots of things aren't working correctly anymore).

Nefari0uss commented 7 years ago

@jrmiller82 Mozilla is making the decision to move away from XUL for their extension framework in favour of web extensions. This move will ultimately make it more difficult to customize nearly as much of the browser and will require lots of extensions to be rewritten. Their reasoning is that the current extension system makes it difficult to update the browser with new features as maintaining compatibility becomes increasingly more difficult. I'm not going to discuss the politics or the pros and cons of this move (that's already been done to death) but the fact of the matter is that it's being done. This means that ultimately Vimperator is going to have to be rewritten essentially from scratch to work with web extensions. This means for a while it's going to be rather painful as either we won't be able to use the extension or that whatever version works with the new ecosystem will be very lite in terms of feature set. Ideally the growing pains of WebExtensions gets re-mediated over a period of time and we have a new ecosystem, a new code base to work with (with whatever architectural improvements starting over can provide), and hopefully (near) feature parity in due time.

fakedrake commented 7 years ago

So maybe someone who knows the ins and outs of vimperator and has some idea of what are the priority features should break down what rewriting vimperator would entail so we can get going. I was looking at the XUL/XPCOM vs WebExtensions article and it looks like very few of the useful features of vimperator are translatable to WebExtensions.

jrmiller82 commented 7 years ago

So in the meantime, is there a vim-like plugin that does work with the new WebExtensions? Or gasp, will I have to use Chrome and vimium or something awful like that? :-1:

fakedrake commented 7 years ago

I am currently using conkeror and IceCat+vimperator trying to decide which one is a better short term solution. Conkeror is barely winning for now...

James Miller notifications@github.com writes:

So in the meantime, is there a vim-like plugin that does work with the new WebExtensions? Or gasp, will I have to use Chrome and vimium or something awful like that? :-1:

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/vimperator/vimperator-labs/issues/705#issuecomment-279270942

jrmiller82 commented 7 years ago

@Nefari0uss Forgot to say thanks for your detailed response. I wasn't trying to spark a political debate. I realize that change happens in the upstream always. :)

Nefari0uss commented 7 years ago

@jrmiller82 No problem! As of now I have VimFX on Firefox as a backup since Vimperator doesn't work with Nightly.

jrmiller82 commented 7 years ago

@Nefari0uss I'll have to check that out.

dschrempf commented 7 years ago

Thanks for the reference to the discussion of the folks using pentadactyl. I would like to add a reference to a related VimFX issue akhodakivskiy/VimFx#860. I like the argument of the VimFX author.

I prefer Vimperator, because I cannot work with a command line that takes nearly half a second to show up but I believe it makes a lot of sense to join forces.

SammysHP commented 7 years ago

Firefox 57 will be released two weeks earlier: https://wiki.mozilla.org/index.php?title=RapidRelease%2FCalendar&diff=1162727&oldid=1160751

pauldub commented 7 years ago

Vimperator is the only reason I use Firefox, if a there is a call for help I would be happy to contribute. I have started thinking about this for some time and I do not know what I would do if vimperator stopped working. My requirements for a web browser are quite narrow and I even started writing a GUI using CEF with the goal of implementing only the minimum functionality I need from a browser. I'd rather contribute to porting Vimperator ;)

maxauthority commented 7 years ago

I fully agree with the others, that starting a Vimperator web-extension should happen in a new repository, and once we have some people being commited in doing that, we shall create a vimperator-web-extension repository in https://github.com/vimperator/ just like we did for the (potentially) new home page and syntax highligthing.

And while on the one hand it'll be sad to see the full XUL vimperator gone, on the other hand it'll free us from the Firefox backend, and people can choose freely between Firefox, Chrome, Edge or Opera, as they all support WebExtensions.

That said, it still requires somebody to do the first step and start a basic framework/skeleton, and be it just with very simple commands like h/j/k/l, and then adding features back in one by one. Probably it'll be easier to just use Vimium (for Chrome) as a base and make it work on Firefox.

mkoskar commented 7 years ago

@dschrempf

I prefer Vimperator, because I cannot work with a command line that takes nearly half a second to show up ...

I believe what you are talking about is built-in "Developer Toolbar" (Shift-F2). VimFx doesn't have its own command line but just focuses browser's urlbar.

dschrempf commented 7 years ago

@mkoskar Yes, this may be what I mean. The bar that I am talking about shows up when I press :.

SammysHP commented 7 years ago

I can summarize what is most important for me:

Commandline/statusbar
o, O, t, T, w, W – they must open something like the completion in Vimperator.
f, F and ;y to follow and copy links
/ with the current behavior, the normal findbar doesn't work for me
y and p to copy and paste URL
Mappings that can override Firefox keybindings like Ctrl+Q
b to show and switch buffers
:set gui=none – just me, the content and a statusbar
Some kind of NoScript integration. Showing the popup would be enough.
Keyboard navigation of course (h, j, k, l, gg, G)
Open an external editor (Ctrl+I) would be nice
Commands to change preferences (:set! …)
Tab commands (d, u, :tabdetach, :tabmove)
Passthrough mode (Ins)

xdml commented 7 years ago

Has anyone checked https://github.com/shinglyu/QuantumVim?

Looks like @shinglyu already started similar project.

shinglyu commented 7 years ago

@xdml Thanks for mentioning. I've implemented a small set of features that is critical to my daily use, but I haven't put much effort into making it a consumer grade product. I'd appreciate if anyone can help by submitting PRs, bugs or feature requests.

shinglyu commented 7 years ago

FWIW, I found fa few issue during my implementation of QuantumVim:

WebExt lacks a way to add global hotkey, you must add "Ctrl+" or "Shift+", and they sometimes conflict with the built-in hotkey. I have to use a page script to capture user hotkeys, but that's not very reliable.
For features like :open, I just want to focus on the URL bar and select all. I already filed a bug but for now I have to implement that in the Add-on SDK. So people have to install two add-ons, one in WebExt and one in Add-on SDK.

xdml commented 7 years ago

@shinglyu Do you think it will be possible to implement command line instead focusing address bar or use awesome bar as command line? My preference is to use gui=none in Vimperator, so willing to have command line in future as well. And that's why I never fell in love with Chrome/Chromium, because of Vimium, there was no command line, and often it happened, that key strokes has not been detected, at the time I played with it.

mavaa commented 7 years ago

If I'm not mistaken, we're not allowed to change the GUI in any way using WebExtensions, right? It's a real downer if I can't have my gui="none", but I guess I/someone could fix that by hacking a bit at the firefox browser itself(?)

Regarding the current status line, would an alternative be to inject a fixed div/element into each open tab and have it stick to the bottom of the screen? Made a quick mock-up here to illustrate: https://marza91.github.io/vimperatortesting/

It would mean that there's a separate status-line being injected into each open tab/page when it loads, and I guess it could potentially mess with the styling of some pages(?), but if there's no alternative, I think it would be better than nothing?

You could (maybe?) do this with command line as well, if we put an input-element in the div, but I'm not sure if there are better ways of doing this in WebExtensions and/or if it would work. I'm going to try to learn some more WebExtensions stuff if I can find the time, and maybe make a statusbar/command mode prototype.

I could try to put this into the QuantumVim project (@shinglyu) as well, but for now, I can't really commit too much time into this. If no one have the time right now to take on a "project lead" role in the WebExtension port, maybe it's just as well if everyone tries to make the part they want the most themselves, and then we can look at merging stuff together later? It would require some coordination here or someplace else though, so we don't end up with 15 implementations of the same thing.

I'm not an expert at any of this stuff, just throwing out some ideas here, feel free to shut them all down :)

mkoskar commented 7 years ago

@marza91 Have a look at cvim Chrome extension. I does something what you are describing pretty well actually. I suppose Vimperator could take inspiration there, although it's pretty obvious since it seems like the only viable option anyway.

protrolium commented 7 years ago

The switch to webExtensions (and deprecation of XUL) in November is upsetting, since I am a user who thinks status bar/command-line + set: gui=none IS ESSENTIAL. In addition to everything @SammysHP mentioned, I also use [[ , ]] very often, as well as my own qmarks and a (for bookmarking).

fakedrake commented 7 years ago

My 2 cents: I had some free time and checked out the chrome source for building the UI. It looks like at least there there is not much room for modification. The layout is hardcoded in C++. Here are some interesting positions in the code:

(Edit: I expect that since this is the case in chrome, firefox devs will not be bending over backwards to make UI configurability possible)

Also I was thinking that for the non-UI stuff that we can't currently do without XPCOM (eg edit textfield in external editor) we could make a daemon in nodejs/python/go/language-du-jour and optionally interact with it over http to unlock that functionality.

wcunning commented 7 years ago

It seems to me that the right way forward is to get some api requests in as soon as possible. Given that thread already some discussion of the required keyboard shortcuts for VimFX, and I would bet on one for UI requirements for TreeStyleTabs, it would make sense to extend those requests or contribute to those apis to get what this project needs.

https://bugzilla.mozilla.org/show_bug.cgi?id=1215061

fakedrake commented 7 years ago

Are you aware of any existing similar thread addressing the UI (a quick search didn't yield anything)

wcunning commented 7 years ago

https://wiki.mozilla.org/WebExtensions/Future

This sorta hints that they know it's an issue, but I'm not finding much either. I assumed that there would be something after reading about the VimFX API request. Still, I think that it shows at least a theoretical willingness to expose an API for those features, since Vimperator and TreeStyleTabs both need something in the same vein and are popular among the userbase.

sirn commented 7 years ago

https://bugzilla.mozilla.org/show_bug.cgi?id=1215064

There's this one about implementing a custom toolbar. Nothing about hiding the toolbar, though.

vimperator / vimperator-labs

Port to WebExtension #705

Chatroom