Changes for 1.9.0

[ssborbis]

• Phasing out auto importing via native app by using new browser.search API in FF 63+. OCSEs (One-Click Search Engines) are imported into CS on launch. They cannot be edited from the search engine manager, however, unless they are imported using the import search.json.mozlz4 button. OCSEs are tagged with a marker in the manager. On search.json.mozlz4 import, nodes pointing to OCSEs will be updated to reference the imported engine, making them editable.

• Search engines can be re-ordered from the Toolbar search and Quick menus via drag&drop @hg42

• Add-on size has been reduced 20% by trimming some legacy code, compressing icons, etc

• Added Remember Search History to Toolbar options.

• Hotkeys can be set for bookmarklets in Quick menu

• Sidebar with opening widget. Widget can be moved left or right, and offset relative to the top of the page via dnd. Widget responds to text dnd. (dnd not available in Chrome)

• Search engine tiles respond to dnd text (dnd not available in Chrome)

• Toolbar search tile formatting matches quick menu tile formatting. FF has a width limit to browser_action windows and so may not accommodate a high number of search engine columns

• web_accessible_resources removed from FF (needed in Chrome)

• Added option to disable Auto Quick menu when selecting more than (n) characters

• Added option to set the initial opacity of the Quick menu. Most useful for auto opening ( ~0.2 shown, default 1.0 )

• Added user-defined stylesheets

• Added option to disable menu animations

• Added lite / dark themes

Fixes

• Toolbar search hides rows if entire row is made up of hidden tools

• Toolbar close on click not working

• Using arrow keys in Quick menu with single column/labels applying class incorrectly causing large icons and missing text

• Context menu displays nodes with duplicate ids

• Toolbar shows search bar when Quick menu search bar position set to hidden

• Adding a new folder opens name for editing

• Short timeout (50ms) in auto opening to filter some accidental double-click selection events

• Overflow / scrollbar handlers for Toolbar Search / Sidebar

• POST engines work in Chrome ( v1.9.1.1 thanks ufo1024k )

---

things to do: rewrite help pages

[hg42]

for a while I used the release version and didn't edit much. Unfortunately my userChrome.js containing

AddonManager.installTemporaryAddon(new FileUtils.File("/z/src/firefox/ContextSearch-web-ext/src"));

to autoload the beta doesn't work any more. So I am testing via temporary loading.

my observations

• the drag and drop in the manager works awesome now. I think the method using the line as indicator is the best solution. This way nothing is jumping around (which sometimes leads to curious situations) and it's totally clear where the drop will happen. This feels very smooth and natural now
• thanks for implementing drag and drop directly in the quick menu, this already helps a lot
  • dragging to the left works as expected
  • dragging to the right works unexpected, I have to drop on entry x+1 to have it inserted at x
  • dragging into a folder and out of a folder would be nice (probably a simple drop an the folder or on the back button would append to the destination folder, or with as a maximum wish and more work the destination folder would open when hovering over it)

I had some problems importing from search.json.mozlz4, but I am not sure what happens exactly. It must have to do with hidden items I deleted from my firefox search box. Also, I am not sure how CS interacts with the firefox search engines list. Am I right to think the term OCSE stands for each firefox search engine available in the search box or the location bar? Because if I import the file, I only get two orange "FF" marks, but nearly all my engines live in both lists. Though most of them are in subfolders, at least some of them live at the top level and are still unmarked.

[hg42]

I am now back to firefox 61, where the autoload of the beta from a repo works.

Toolbar search hides rows if entire row is made up of hidden tools

something isn't like I would expect... I assume "hidden tools" means enabled tools, that are shown in toolbar menu as empty buttons. The goal is to make quickmenu and toolbar search similar in structure.

3 tools enabled, 2 disabled 3 column quickmenu, 6 column toolbar search -> hidden -> ok, both start engine at 1st button 6 column quickmenu, 6 column toolbar search -> shown -> ok, both start with engine at 4th button 6 column quickmenu, 3 column toolbar search -> hidden -> ok, empty column removed 3 column quickmenu, 3 column toolbar search -> shown -> NOK, empty row shown

4 tools enabled, 1 disabled 4 column quickmenu, 4 column toolbar search -> shown -> NOK, empty row shown

I guess the pseudo code should be:

if ((n_enabled_tools == n_cols_quickmenu) || (n_enabled_tools == n_cols_toolbarsearch))
    hide_emtpy_buttons(); 

[hg42]

oh, nice, I see the tools can be moved to the bottom and this removes the empty buttons, too...

[ssborbis]

whoops, listed some fixes that weren't in the repo yet. Just pushed a bunch of changes

[ssborbis]

dragging to the right works unexpected, I have to drop on entry x+1 to have it inserted at x dragging into a folder and out of a folder would be nice (probably a simple drop an the folder or on the back button would append to the destination folder, or with as a maximum wish and more work the destination folder would open when hovering over it)

On the to-do list

something isn't like I would expect... I assume "hidden tools" means enabled tools, that are shown in toolbar menu as empty buttons. The goal is to make quickmenu and toolbar search similar in structure.

I think that's fixed in the last commit. I'll check though

I had some problems importing from search.json.mozlz4, but I am not sure what happens exactly. It must have to do with hidden items I deleted from my firefox search box.

Last push has some code to filter out bad nodes in the tree on launch. I'll add it to do so after importing search.json.mozlz4 too. (edit) looks like it does already

I am now back to firefox 61, where the autoload of the beta from a repo works.

Am I right to think the term OCSE stands for each firefox search engine available in the search box or the location bar?

I'm using OCSE as shorthand for the OpenSearch engines installed to Firefox's search bar. The ones that you can manage through Firefox->Options->Search. The new browser.search API allows me to access those on a limited basis without having to import search.json.mozlz4. I can read names and icons, but that's it, so editing parameters,icons,etc can only be done by importing the search.json.mozlz4 file. This is useful for FF63+ users who:

• have OCSEs installed before adding ContextSearch and can now access them without needing to import search.json

• install OpenSearch (OCSEs) through sites like mycroftproject and can access them immediately without having to import search.json

My plan is to more or less ditch the automatic importing completely. I think most FF63+ users won't notice the difference. How do you feel about it? Do you add a lot of search engines from places that mycroftproject that you want to edit afterwards? I think that's the only scenario where it becomes a hassle to not have the auto import.

[hg42]

I mostly use "add custom search" to add new engines, because I almost always want special search options, which usually works well. So, no, I don't really need to import OCSEs.

However, I assume, there are users that just want to use predefined search engines from mycroft. Perhaps CS could import such engines from mycroft, may be via context menu or similar.

My intention for connecting to search.json is for another reason. I would want OCSEs to be in sync with my CS engines. This has different reasons, may be I could use CS alone instead. But I often fall back to using the FF search box.

One reason is using drag and drop of search text from a page or from outside of firefox into the search box. DnD doesn't work with CS, and I wish it would. And yes, I know I could use Copy&Paste instead, but I am so used to dragging things around, I can't accept more clicks to do the same thing :-)

Another reason is searching some text from firefox pages like about:config or settings.

My workflow is

• dragging text from somewhere to FF search box and using some well known OCSEs
• using CS uickmenu or toolbar search with the new beloved folders when I need some less frequently used engine
• using CS for something special like bookmarklets, that I still need to test, though most of my beloved bookmarklets were broken by newer firefox versions, because of cross site scripting etc.

[hg42]

will test new commits tomorrow, when I have access to my workstation

[ssborbis]

However, I assume, there are users that just want to use predefined search engines from mycroft. Perhaps CS could import such engines from mycroft, may be via context menu or similar.

Search engines installed from mycroft become OCSEs, which as of 1.8.2 will be added to CS automatically on FF launch, or by opening CS options. These are the engines tagged with the FF. Usable with CS, but not editable unless imported. This should satisfy the needs of most users.

One reason is using drag and drop of search text from a page or from outside of firefox into the search box. DnD doesn't work with CS, and I wish it would. And yes, I know I could use Copy&Paste instead, but I am so used to dragging things around, I can't accept more clicks to do the same thing :-)

If there was a way to do it in the API, I'd see it done, but it looks like the browser_action event list is limited to 'click'. I don't think you can even trigger it to open programmatically or I'd trying opening it on document dnd events. I suppose the functionality could be somewhat simulated by injecting an absolute div/input in the top right corner that responds/expands/opaques to dnd/mouseover. Interesting concept..

[ssborbis]

My workflow is dragging text from somewhere to FF search box and using some well known OCSEs

Wrote up a quick widget to handle dnd. Also opens/closes on click. Effectively, a replacement for the browser_action with dnd support. I suppose if I kept this code, I could leave it to the user to decide which to use. Some people might want to save some toolbar real estate.

[hg42]

Search engines installed from mycroft become OCSEs, which as of 1.8.2 will be added to CS automatically on FF launch, or by opening CS options

ok, then there seems to be no need to import them

As a summary, I have two possibilities:

• If I always use CS and don't need the FF searchbox anymore, then I would ideally want to have everything in CS.
• If I still use the FF searchbox, I ideally want to have CS and OCSE synced and use the more comfortable for editing (which is CS, obviously)

From what I read, the second variant isn't really possible (export of the whole flattened CS structure to OCSE, including the sequence).

And for the first variant, drag and drop becomes the main issue.

If there was a way to do it in the API, I'd see it done, but it looks like the browser_action event list is limited to 'click'

I think we already discussed that topic...

[hg42]

Wrote up a quick widget to handle dnd

wow, that's awesome... this would probably kill my FF searchbox

some thoughts:

• where can this widget live? is it restricted to the page? or can it be everywhere?
• if only page, can it live in an internal firefox page like settings or about:*? what about dialogs? (btw. I noticed about:config doesn't work anymore)
• the upper right corner is often occupied by toolbar buttons (other corners might also be used, e.g. by a "to top" button)
• perhaps the box could be draggable and I think the position should then be stored as a percentage and/or a distance to the nearest corner, so it could stick to the bottom right, for example, even when resizing the window
• each search engine could be a drop target
• dragging over a folder could open it underneath (and close it again over the back button)

[hg42]

oh, surprise! I wasn't aware, that the dnd code was already pushed to github.

• as expected, the dnd widget does not work on internal pages like e.g. add-ons manager (I still don't get why this is restricted by mozilla)
• interesting: dragging text from an internal page to another tab switches the tab and I can drop on the widget
• because a folder is a drop target now (nice! btw. dropping on back button is missing), I can drop a text on it, but it's interpreted as url
• I think drop on the text field should replace the text
• the widget can now remain open like a sidebar most of the time
• because of that I was trying to reduce the width of the widget, when I noticed that this width is fixed. Now I understand, why the buttons in the toolbar search are wider than in the quickmenu with my 6 column setup :-) I think I noticed that fixed width on other add-ons, too, is it impossible to fit the widget to the buttons? at least from a quick look the code seems to set the size

[ssborbis]

perhaps the box could be draggable and I think the position should then be stored as a percentage and/or a distance to the nearest corner, so it could stick to the bottom right, for example, even when resizing the window

If I keep the code, I'll definitely make it configurable. It takes a bit of extra code to get the positioning correct due to scaling relative to page zoom, but I had it figured out for the qm, so this shouldn't take much.

each search engine could be a drop target

Seems reasonable

dragging over a folder could open it underneath (and close it again over the back button) dropping on back button is missing

Work in progress. I'm thinking about how to handle dnd on folders in general. Hover more than .5s to open folder/back button? I'll have to play around with some code. Not sure how much alteration that would require. A simple drop on the back button would be easier but less robust.

I can drop a text on it, but it's interpreted as url

Can you elaborate? (edit) nvm, i see what you mean

I think drop on the text field should replace the text

I agree. I'll set it up

because of that I was trying to reduce the width of the widget, when I noticed that this width is fixed. Now I understand, why the buttons in the toolbar search are wider than in the quickmenu with my 6 column setup :-) I think I noticed that fixed width on other add-ons, too, is it impossible to fit the widget to the buttons? at least from a quick look the code seems to set the size

I'm reusing the code/settings of the browser_action for the widget. I tried to make it look as close to my Firefox search bar as I could. I was limited in width/height for the Toolbar search ( i forget exactly what the limit was. it could be resolution-dependent. i never checked ), but there is much more freedom to set dimensions for the widget being an injected iframe. I could have made it reflect the Quick menu vs the Toolbar search just as easily. Somewhat arbitrary decision I suppose.

[hg42]

dnd on folders in general. Hover more than .5s to open folder/back button

the simple version would just append (not open on hover). With hovering it would allow to drop at an exact position, but the changing widget leads to difficult situations sometimes. I am not sure how many users want to use it, it's probably better to make it opt-out (or opt-in?).

limited in width/height for the Toolbar search

ok, but it's fixed width, so each button is w_button = w_widget / n_cols apparently it could be:

w_widget = min(wmax_widget, w_button * n_cols);
n_cols_real = int(w_widget / w_button);
w_widget = w_button * n_cols_real;

btw. making it similar to my searchbar, it would have 23 columns :-)

Ideal format is a square, because this results in minimal mouse travel. For the side bar I want it more narrow.

[ssborbis]

each search engine could be a drop target

Working example in latest commit

I think drop on the text field should replace the text

My code needs a little tweaking to handle some dragenter/dragleave repeat behavior, but I have it set up to replace text if dropped within 0.5s, otherwise append text. Text selection indicates replace vs append.

btw. making it similar to my searchbar, it would have 23 columns :-)

I'll look into a better width handler tomorrow

[hg42]

each search engine could be a drop target Working example in latest commit

many thanks, with this change CS is now ahead of the FF searchbox, where I need drag+drop+click. And it's still on par for a search engine in a folder, because it opens when dropping on it. Good work.

I think drop on the text field should replace the text replace text if dropped within 0.5s, otherwise append text

interesting, so we have both worlds... it's also very good that a move reactivates the full selection

there seems to be a race condition (at least for the slow human mind). Sometimes the text is inserted despite clearly seeing the selection. I'll do some more tests...

[hg42]

I tried using a hoverTimer of 1000ms. This works much better for me. 1000ms can only happen with my intention, while 500ms sometimes triggers without my intention (probably when being slow, and I think I'm usually very fast with my mouse, most friends cannot use my speedy mouse settings).

[hg42]

I had some more thoughts about replacing FF searchbox:

• FF searchbox is big and therefore it's easy to click on it, it's also good for a drop target
• curiously, I never use a keyboard shortcut to open FF searchbox (I don't even know it), but I guess some users do. I wonder, why CS toolbar search doesn't have a shortcut. The sidebar would need one, too.
• when I tried using different search mechanisms on some internal FF page like an add-on description, I noticed, that the CS toolbar search doesn't import the selection to the text field. I think to remember this once worked... I was a bit surprised when it also didn't do that on a normal page
• I hope all my comments don't push you too hard...I don't want to bother you...actually the extension is already doing a lot of nice things

[ssborbis]

I tried using a hoverTimer of 1000ms. This works much better for me.

Good to know. I'll try it out

curiously, I never use a keyboard shortcut to open FF searchbox (I don't even know it), but I guess some users do. I wonder, why CS toolbar search doesn't have a shortcut. The sidebar would need one, too.

I left this feature out of the brower_action because it required a hardcoded key combo and I didn't want to interfere with other gestures/addons inadvertently. A key combo to open the widget/sidebar on the other hand can be configured by the user. That's a nice feature that I'll take advantage of by moving the shortcut setting from the quickmenu (pretty useless) to the widget/sidebar.

when I tried using different search mechanisms on some internal FF page like an add-on description, I noticed, that the CS toolbar search doesn't import the selection to the text field. I think to remember this once worked... I was a bit surprised when it also didn't do that on a normal page

I'll check on that. Some code may have been lost when I was merging the backends for the 3 menus into one.

I hope all my comments don't push you too hard...I don't want to bother you...actually the extension is already doing a lot of nice things

Not at all. I appreciate all the testing your doing.

[hg42]

the whole userChrome.js is frustrating. Mozilla seems to remove it more and more. But currently it works with https://github.com/Sporif/firefox-quantum-userchromejs so I can autoload the github version of CS even on Nightly 65.0a1

[ssborbis]

the whole userChrome.js is frustrating. Mozilla seems to remove it more and more.

I couldn't get it working either. I've been using loading it manually via a toolbar bookmark to about:debugging & a shortcut to the manifest on my desktop

[hg42]

I tested the files from https://github.com/Sporif/firefox-quantum-userchromejs for a few days now and they work fine (at least the mechanism for loading scripts and css, not sure about xul). However this is still kind of deprecated and I think it will finally go away forever or at least the possibility to tweak the UI will be blocked in some way. Mozilla is even discussing removal of the main javascript object for the UI.

This binds userChrome.xml (not sure how the mechanism works):

userChrome.css

@namespace url("http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul");
@namespace html url("http://www.w3.org/1999/xhtml");

@namespace url(http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul);

#PanelUI-menu-button {
    -moz-binding: url("userChrome.xml#js");
}

And this loads all the

• scripts chrome/*.uc.js and userChrome.js
• styles chrome/*.css
• agent styles chrome/*.as.css (I use this for scroll bars, I believe it's for kinf of internal elements)
• xul chrome/*.uc.xul and userChrome.xul would be loaded, but loadXUL gives an error

userChrome.xml

<?xml version="1.0"?>
<!-- Copyright (c) 2017 Haggai Nuchi
Available for use under the MIT License:
https://opensource.org/licenses/MIT
 -->

<!-- Run userChrome.js/userChrome.xul and .uc.js/.uc.xul/.css files  -->
<bindings xmlns="http://www.mozilla.org/xbl">
    <binding id="js" extends="chrome://global/content/bindings/toolbarbutton.xml#menu">
        <implementation>
            <constructor><![CDATA[
                if(window.userChromeJsMod) return;
                window.userChromeJsMod = true;

                var chromeFiles = FileUtils.getDir("UChrm", []).directoryEntries;
                var xulFiles = [];
                var sss = Cc['@mozilla.org/content/style-sheet-service;1'].getService(Ci.nsIStyleSheetService);

                while(chromeFiles.hasMoreElements()) {
                    var file = chromeFiles.getNext().QueryInterface(Ci.nsIFile);
                    var fileURI = Services.io.newFileURI(file);

                    if(file.isFile()) {
                        console.log("----------\\ " + file.leafName);
                        try {
                            if(/(^userChrome|\.uc)\.js$/i.test(file.leafName)) {
                                Services.scriptloader.loadSubScriptWithOptions(fileURI.spec, {target: window, ignoreCache: true});
                            }
                            else if(/(^userChrome|\.uc)\.xul$/i.test(file.leafName)) {
                                xulFiles.push(fileURI.spec);
                            }
                            else if(/\.as\.css$/i.test(file.leafName)) {
                                if(!sss.sheetRegistered(fileURI, sss.AGENT_SHEET))
                                    sss.loadAndRegisterSheet(fileURI, sss.AGENT_SHEET);
                            }
                            else if(/^(?!(userChrome|userContent)\.css$).+\.css$/i.test(file.leafName)) {
                                if(!sss.sheetRegistered(fileURI, sss.USER_SHEET))
                                    sss.loadAndRegisterSheet(fileURI, sss.USER_SHEET);
                            }
                        } catch(e) {
                            console.log("########## ERROR: " + e + " at " + e.lineNumber + ":" + e.columnNumber);
                        }
                        console.log("----------/ " + file.leafName);
                    }
                }

                setTimeout(function loadXUL() {
                    if(xulFiles.length > 0) {
                        document.loadOverlay(xulFiles.shift(), null);
                        setTimeout(loadXUL, 5);
                    }
                }, 0);
            ]]></constructor>
        </implementation>
    </binding>
</bindings>

I added some error catching, that prevent script errors to stop loading completely (other scripts and styles)

This line loads CS, I guess it could also be in it's own file (e.g. contextsearch.uc.js):

userChrome.js

AddonManager.installTemporaryAddon(new FileUtils.File("/z/src/firefox/ContextSearch-web-ext/src"));

I have lost a lot of features because of FF Quantum.

Some of them are now reimplemented as Webextensions (especially Tree Style Tab and CS). There are still some conceptual issues like context menu on tree style tabs not being the same as for normal tabs, which means I cannot switch off the normal tabs (I reduce their size instead) or CS not really working on internal pages.

Some of my former tweaks seem to be impossible (Locationbar Enhancements [Bradcrumbs etc.], Search WP [buttons for search words inside searchbar]).

I also used userstyles to heavily tweak the UI, like multirow bookmarks with small buttons, etc. Some of these extensions and tweaks can be reimplemented as javascript. But if they remove that, too, I will probably use another browser.

However, no browser has all the main features I need.

[ssborbis]

I have lost a lot of features because of FF Quantum

The sole reason I'm a developer now.

I still can't get userChrome.js working. Does it only work in the developer edition?

No worries though. I'm used to doing it the old-fashioned way. I'll play around a bit more and see if I can't get it set up.

[hg42]

Does it only work in the developer edition?

I tested on some older version, too, but there should be a minimum version, because in each version slight things changed. May be it was FF 61 or FF63 I started with.

[hg42]

Does it only work in the developer edition?

I tested with firefox-esr on Linux, which is 60.3.0esr and userChrome.js works. I am writing this from ESR.

However, switching back and forth FF versions while using the same profile directory seems to corrupt some databases, especially the CS menu is completely lost (but export/import works) and some extensions are incompatible. I noticed this before, but don't know why this happens. I guess storage.js files are converted when upgrading. At least they are all changed when switching (checked with a diff program).

[hg42]

here are my currently used files:

firefox-quantum-userchromejs.zip

[hg42]

Could you create the sidebar at the same position like the drop target? Because I cannot leave it open at the top right corner, because there are always buttons. Not sure how to do this at the bottom, probably bottom of sidebar = bottom of drop target. Dragging the sidebar would also be nice...

[ssborbis]

here are my currently used files: firefox-quantum-userchromejs.zip

That did it! Thanks man.

[hg42]

(˵ ͡° ͜ʖ ͡°˵)