Change jsm to WebExtension (jsm deprecated with FF 57)

[AWolf81]

As mentioned on MDN jsm will be deprecated soon and it is not recommended for new Addons - see here.

Instead WebExtensions should be used and that will also make it compatible with Chrome, Safari (and probably IE).

WebExtensions are supported with FF releases 51 and above. FF 51.x is released and available.

This is a larger task and it will be done on branch migration-webextension.

[feinstaub]

I got some info from Mozilla:

On 07.04.2017 19:11, Firefox Add-ons Team wrote:

Hello add-on developer,

WebExtensions https://click.e.mozilla.org/?qs=f92deee189926bef7c3a0bbaad83cefc2ab6a81909718d79bcd32004333b10dd9be3fffa2bf0118c7b88ab2a0a9716698922b11390cabceb are the new standard for add-on development in Firefox, and there are some new things (and a few old ones) to help you update your legacy add-on to the new standard. Legacy add-ons will no longer load in Firefox 57, which will be released https://click.e.mozilla.org/?qs=f92deee189926befb4a1f05ef83ea7b3b7c4d399d0b929720d3941ef5984277a683e746c0e6d7a8573479616fb6a7c2e70aed2d9fcfe846ein November, so we urge you to update your add-on as soon as possible.

Embedded WebExtensions

Embedded WebExtensions https://click.e.mozilla.org/?qs=f92deee189926bef7e74e48cc351c05ded6ced40221c36e89f82a107ec8ee8dcb302174474ffdaa343b8ab86f3df01ce52296f93b8180924 combines two types of extensions in one, by incorporating a WebExtension inside of a bootstrapped or SDK extension. This is helpful as a step towards transitioning to WebExtensions, because it allows you to port your user data to a format that can be used by WebExtensions. You can read about AdBlock’s recent experience https://click.e.mozilla.org/?qs=f92deee189926beff65d8372f83895bcfe747a91a970ae1831e3f90a66d062d852a00321bf1c695aefb7e969ad262259d8e3a7a76f5a4f46with this on our blog.

WebExtensions APIs

If you intend to transition to WebExtensions, but the APIs you need aren’t yet available, there are a few ways you can help expedite their availability. From simply requesting it, to advocating for it in a public meeting, to building it yourself, there are options https://click.e.mozilla.org/?qs=f92deee189926bef10cb0dcca701814d8885384ed43ca43675b1b3fe9668f914bfea158037680f7eb32ad3033e945480b1e6be07bd1b0868to get you what you need.

Testing Beta Versions

If you want to test a WebExtensions version of your add-on with a smaller group of users, you can make use of the beta versions https://click.e.mozilla.org/?qs=f92deee189926bef06171ce15b4824c397101ccc73cefd14da7a5e54f15135ac1ff218edb95cd319f92bb3d9a8f96c01202252aa1d85ba71 feature on addons.mozilla.org https://click.e.mozilla.org/?qs=f92deee189926bef4118ebd9469d07b77b94028b110b87f16c4bc8f3dddc60e4df020c4eef6905d2e2dc3853b9e7ca2e614877ab3094e5a2 (AMO). This feature lets you have pre-release beta versions signed and available to Firefox users who want to give them a spin.

New Ways to Get Help

More personalized support is available during our new Office Hours https://click.e.mozilla.org/?qs=f92deee189926beffccceaed05f8654c9d002e50a60274225dff01e2ccc603e5d1690690cc1704f5797f0c898d5485f638fb4fe81656f65e , when volunteers are available in IRC https://click.e.mozilla.org/?qs=f92deee189926bef870d6f978af977b7437640d70ddf84394919ae79edac56c1ad85ba0bef4028454db0759a3972d7d885a4265035a9ff20and forums https://click.e.mozilla.org/?qs=f92deee189926befdc5ac2291a65864b168b330da7fc3ed28cd6b05c2062ec35f776744ff4286cbeb9e61a2c5329b1f84b96e68ef4bc0b02to answer your questions. The schedule automatically displays hours in your time zone, and lists each volunteer’s area of expertise https://click.e.mozilla.org/?qs=f92deee189926bef7653b9f6bbb3242556dff23ff8c5c7cf742ca467498d7bc1d6da592bdb01f7c38a9e1e3e28859c279449b2a7485750d5 on the bottom of the page.

To reach a broader group of helpers, join the webextensions-support mailing list https://click.e.mozilla.org/?qs=f92deee189926bef16a7b872ebe71f2febff453d45793caf1db93c6146cea80423d58538eb096b4874b100e6e61746d1fd7b9798b75f8f7b , where you can ask questions and see what others are having trouble with.

AMO will include legacy add-ons for a period following the release of Firefox 57, but they may appear lower in search results and with a warning to users.

Thank you,

The Firefox Add-ons Team

addons.mozilla.org

---

/You have received this email because you are a registered Add-on developer on addons.mozilla.org./

Mozilla 331 E. Evelyn Avenue Mountain View, CA 94041 Mozilla Privacy Notice https://click.e.mozilla.org/?qs=f92deee189926bef54096f3ab7cddd16274cb3996dd1f57ee1bf39aaeb62be294535da2448f75f72d2380e28dd9cd4133097704ef7283d8c

[AWolf81]

@feinstaub I think we have a deadline for the transfer to webextension. Planned release date for FF 57 is the 14.11.2017.

So please add a deadline to this issue. I'll try to start coding asap.

If you have time you could help me with the native app part that's needed for the web extension. Especially how it could be installed.

I would create the native app as python script becaue that's pretty portable. An installation routine is required because we need to have a registry entry for the native communication. In the demo that I've created it's done with a batch file but that's just OK for development.

[feinstaub]

I added a deadline to the next milestone.

Do we really need a custom native app or could we just use an existing one (like windows explorer for windows and xdg-open for Linux)?

[AWolf81]

Thanks for adding the milestone.

Windows Explorer and XDG-Open can be used to open the link but for the communication part to the Extension a script is required. I don't like that we need a native app/script but I think there is no other way.

I've modified the example python file from the native communication example app and added the opening logic (tested in Windows).

But at the moment my demo app has an issue with sending the native message. Not sure what's wrong because it was working. Maybe registering the app is not working. I have to do more tests.

Please have a look at my first prototype here.

Once I'm satisified with the repository of the prototype, I'll transfer the code to a branch here so we're having everything in one place.

[feinstaub]

Hi!

1.) We need communication only for feedback if the file could be started or not, right? So, as fallback we could use this method without a native app, can't we?

2.) Thanks for providing the prototype.

I followed these instructions https://github.com/AWolf81/webextension-local-filelinks-prototype/blob/master/README.md under openSUSE Linux.

$ npm install $ npm run dev http://localhost:8080/

src/host/install_host.sh was not executable. I changed it and run it. Can you fix it in the codebase?

Now, I must admit I don't fully understand what should be done to test the addon in Firefox. Could you add a "for dummies" section in the README.md? :-) Sadly, I am not that familiar with current JavaScript development technologies.

[AWolf81]

Hi:

to 1) As of my understanding it is always required because web extensions can not access local files. A native app is like the interfacing layer between Firefox and the file explorer. FF tells the native app - hey open that link in Windows explorer or XDG-open and the native app can also return to FF that file access was OK. It would be great if there would be a way with-out it but I think that the chance is small.

to 2) Why was it not executable? Was just the file attribute +x missing? I'll add that.

Sure, I can add more details to the README.md. But where do you have the problem?

• Can you load the extension in Firefox? There should be an icon in addonbar with a dummy drop down page. Later we can use this for quick access settings.
• Are local file links clickable? This is where I have problems at the moment. It was working to open files but I'm not sure why native communication is currently not working. Or maybe I have trouble with whitepage setting.

Security concerns

We need to add a hash check that the native app is our installed file at every communication so that a replaced file won't work with the WebExtension. Possible security issue with the extension. A virus/ransomware could use this to trigger an app execution if they find a way to replace our original executable. Points we need to check here:

• Is it possible to replace the native app with-out changing the WebExtension installation? E.g. user clicks accidentally on an executable file (it will just copy the virus to the native app location) that a hacker created to try if there is our extension to trigger an action. The hacker could directly trigger an action but if it changes the native app it will look like we're doing bad things.
• Executable file check needs same handling as in addon. I've already added this to the Python script but this needs to be verified.
• Checking the native app hash could be difficult. Because how can you trigger a check of the file with-out file access? With file access from extension you could calculate the hash value of the native app. Firefox needs a way to check if the file is unmodified. I couldn't find more details to native messaging security at Google. Maybe we need to chat with some Mozilla Devs to see if there is a way to check the native app or if it is not that critical - but I think it is.

[feinstaub]

to 2) Why was it not executable? Was just the file attribute +x missing?

yes

I'll add that.

thanks

Sure, I can add more details to the README.md. But where do you have the problem?

I opened the browser - the one I am now browsing with, plus the current Local Filesystem Extension installed - at http://localhost:8080/ and nothing special happens except for seeing the page "Welcome to Your Vue.js App".

Security concerns

Thanks for making these considerations. We could place your thoughts in an extra issue and/or adding a section to the README.md.

[MrWillow]

Hi AWolf81 and feinstaub,

I am curious if you made some progress on this daunting issue. If I understand everything correctly the upcoming 57 release of firefox (in november) will break this useful add-on completely.

[AWolf81]

Hi MrWillow:

I'm sorry there is no progress. I'm pretty busy at the moment and I'm a bit stuck at the prototype. It's probably just an installation issue with the native app but I couldn't find the issue.

Yes, you're right. The release in November will break the add-on and it can't be used anymore.

Thanks for asking and reminding me at the issue.

I think it should be possible to get it working until November but probably we need to remove some features. So we can focus at the core features - opening links directly, directory open with the icon and whitelisting.

[feinstaub]

@AWolf81: thanks for the assessment. As for me, nice-to-have feature removal is ok. The main thing is that the addon is usable at all.

[feinstaub]

If you got something I'll test it.

[MrWillow]

Good to see a prompt response from you both! I totally agree: Get something up that works is much more important than nice-to-have features. If I may suggest:

• Make sure network shares are supported (very common in office like environments)
• whitelisting (with port matching). I saw that things like http://myname:82/ do not work when whitelisted in the latest version. (Where 'myname' isn't something like www.bla.com, but something that resolves locally on the nameserver used)

Good luck! I am looking forward to an updated version.

[AWolf81]

@MurzNN Thanks for pointing me to Open with repo in issue #95. I can learn more about how to create webextensions.

I need to check how their installation script is working and how they trigger it. Maybe parts of it can be used here. After some searching in their source code it's clear that they're not having an installer and the user needs to install Python on their own. Also the install script needs to be triggered manually.

An installer is required because otherwise it's too complicated for most of the users to setup the extension. See comments here.

At least a batch file with a python executable e.g. with py2exe that's copied to user's home directory is required to install the native app part.

[AWolf81]

Some infos to current look and feel (I think it's OK but if there should be something improved please let me know): Addon settings page

Test page and popup menu in Addon bar The menu has only a link to the options page (for faster access to the settings) at the moment but I think we can add more feature here later (e.g. add current page to white list or add a link to a help file)

I think it's pretty similar to the previous version just the settings page looks differently but I think it's OK. I'd like to use the css styling from Firefox but wasn't sure how to do it - so I created my own styling.

I'll skip the context menu for now as I think it's optional and we can add it to the next version.

Next I need to create the installer for the native host app for Windows. Once this is done I'll create a package of the extension and post it in the issues. After that I'll move to Unix testing - nothing done yet as I'd like to finish Windows development first.

[feinstaub]

Thanks for the status. Looks fine. Some minor remarks:

• The new version would be 0.99.57
• The title on the popup on the menu bar should be consistent to the Extension's name. So the title would be "Local Filesystem Links"

OK to all other items you mentioned.

[MrWillow]

Thank you very much. The screenshots look good. I am looking forward leaving firefox 52.5 ESR behind me. :-) Keep us posted please.

[AWolf81]

I have the extension working under Windows. Installer is ready and working. A smaller bug that I need to fix is that the Notifications are not shown at the moment.

Next I need to check Linux and make it work there.

The extension can be tested like following (it's easier to install later - it's just a bit more complicated for the test version):

• Download the zip file and extract it to a destination at your machine
• Then open FF & add about:debugging as URL + hit enter and load the manifest.json in the extracted directory
• Next click on the addon bar icon and click on show installation guide
• Now download and install the app with the windows installer (unsigned). You have to click 'show more details in the Windows Defender prompt and click install anyway. (Maybe you have to check install VS redis. at the first dialog if you don't have it - maybe I'll add a check in the installer later).
• The addon is now temporarily installed and can be tested. The app is uninstallable in the Windows program manager.

The install flow should be later like following:

1. Install the extension from Mozilla store
2. Extension shows installation guide
3. Manually install the native app.

@feinstaub Thanks for your remarks. I've added them. I have some points that I'm not sure how we should handle them - maybe you're having an idea how to solve: In the migration branch I've a dist folder and an artifacts folder. They're both build directories and are created from the src directory with npm scripts. What do you think should we add them to the master branch later or should we .gitignore these folders? Maybe we could add them in a release branch but I'm not sure.

What do you think should I create a PR for the migration branch or should I wait until I have it working in Linux?

[feinstaub]

In the migration branch I've a dist folder and an artifacts folder. They're both build directories and are created from the src directory with npm scripts. What do you think should we add them to the master branch later or should we .gitignore these folders?

The build directories should not be part of the repository. They are generated from source; so why should they be checked in? Or what are your thoughts why these folders should be kept?

Maybe we could add them in a release branch but I'm not sure.

I would not place binary releases into the source repository. One could create a separate repository or use the github downloads feature. Normally, this would not be necessary because the binary release is published on the Mozilla website.

What do you think should I create a PR for the migration branch or should I wait until I have it working in Linux?

Please go ahead with the Windows-only version. I think it would be good to have a Quantum-ready version as soon as possible. Unless you feel that this step would strongly weaken your future efforts with the Linux version.

[AWolf81]

/dist folder topic Yes, you're right. The generated files shouldn't be checked-in in source repo. (I'll add dist to .gitignore) but I've thought it would be good to have the binaries somewhere. As it could be difficult to generate the data for the Mozilla binary in one build script like it is now. I think building the extension is the same for all platforms but the native app part will be different for each platform.

Also I'd like to add the native app to the extension bundle, so it's automatically available in the installation guide page. But maybe a link to a download page would be also OK so we could separate the native apps into a separate repository.

So I think it's required to create a repository for the native apps source in a separate repository and use links to the download location for each platform. Then we can build & bundle the extension on any platform and the native apps can have platform specific build setups.

And one repository for the released binaries for the native apps. So we can use it for downloading the installers. You're right extension binary not needed as it is hosted in Moz. store.

Finalizing the extension I'd like to check Ubuntu if it's not taking too much time. I try to finish this in one or two weeks. I think it has to work at least on Windows and Linux. Unit tests can be created for the next release as I think it's OK to manually test the extension for the first release.

@feinstaub Could you please help with the following todos:

• [ ] Check if there is a way to sign the executable files for Windows? I haven't found anything with out paying. I think code signing is required for Windows so that the user knows that the excuteable can be trusted. I'm also not sure if this would be a one time payment or a payment every year.
• [ ] Create the repositories as mentioned for the nativeMessaging host application (source) and one for the released binaries.

[feinstaub]

Check if there is a way to sign the executable files for Windows? I haven't found anything with out paying. I think code signing is required for Windows so that the user knows that the excuteable can be trusted. I'm also not sure if this would be a one time payment or a payment every year.

I skimmed over https://msdn.microsoft.com/en-us/library/ms537361(v=vs.85).aspx "Introduction to Code Signing"

Windows Code signing addresses two issues:

1. "Ensuring authenticity -- Assures users that they know where the code came from."
2. "Ensuring integrity -- Verifies that the code hasn't been tampered with since its publication."

Authenticity: I am not sure if it helps if people are assured that the addon is guaranteed to come from two individuals they don't know anyway. ;-)

Integrity: Do you think it is possible to integrate the native app as binary blob into the addon? Then we could include a download link that points to the integrated file and show instructions of how to install. We would not need to establish and secure a separate download site. Integrity would be established by trusting the Mozilla addon store and the integrity of my Mozilla developer account.

The rest of the document explains a rather complicated infrastructure which I don't fully understand.

On the other hand, I just found this answer regarding code-signing when dealing with GNU/Linux/FOSS software: https://stackoverflow.com/questions/1732927/signed-executables-under-linux/1734637#1734637. Reading this, I think time is better spent to work towards more free software platforms than to engage in binary code-signing with some minor benefits for a non-free platform. Maybe, in the future, code-signing will be mandatory for Windows anyway. I will wait until then.

Are there any good reasons we should do the code-signing as soon as possible?

Create the repositories as mentioned for the nativeMessaging host application (source) and one for the released binaries.

Since the addon only works together with the native messaging host application I think we should keep this code together with the addon's code. Maybe rename the "host" folder to "native-host-application" and move one level up. Then have subfolders "windows-src", "windows-bin", "linux-src", "linux-bin" which contains the source and binaries. In this case it would make sense to me to include the binaries. The *-bin folders would be the source for bundling the native host app with the extension package. We could bundle all platform specific binaries into one extension package. What do you think?

[AWolf81]

Code signing Thanks for your research. Yes, you're probably right with authenticity. :-) I think that we can not bundle the installer into the extension so that it is not showing the warning. At the moment the installer is part of the extension and after clicking on download link you have the file locally and after starting you're getting the warning. (Maybe if it's possible to directly open the executable from the extension it won't show the message but I'm not sure if this is possible.)

I think we don't have to sign the files but it would be better. The installer can be started even it is unsigned but the warning message is a bit confusing.

The topic is really not that easy to fix. So I think we could fix this later. But we should keep this in mind and solve as soon as possible because nobody likes to click away system warning messages and install it anyway.

Host folder Good point. That's also OK and probably easier to manage. This way we can also add the download link to the installer files in the binary directory from Github. The source of the native app seems to be cross-platform as it is a Python script. But the installers will be different. I think it's OK like this (starting in top-level directory):

/native-host
   package.json (not sure if it's required - I'll check if this is working for build scripts)
   /src (contains the Python script)
   /build
     /windows (build related files)
     /linux
   /bin/windows (installation files)
   /bin/linux

I'll create the directory structure like this.

[AWolf81]

Fixed with 0.99.57

[MurzNN]

Please provide in Extension description text on Mozilla website - info that it need to install external application, and link to page with instruction how to install native application.