Intentionally selecting query method

[noctuid]

For example, if I type sl as a query with the intention of it being an intialism, I'd rather all matches (or at least the first ones) be those matched by an initialism. This could be achieved by sorting initialisms first, but this package doesn't do sorting, and that would be overly complicated and slower.

What might be nice if there was some way to mark a query as something specific. I was thinking you could potentially customize different separators for different query types, which would allow doing this without any extra keypresses. For example - could match an initialism, could match an initialism or a regexp, etc. There could be a default if you haven't typed a separator yet. Even better would be if you could set a default per-query number (and as an alist per command). I guess this could potentially have speed improvements as well.

For example, I might set the default for query 1 to be initialism and the default for query 2 to be a regexp. as a separator would not change things from the default. Normally, I would start with an initialism and then add a regexp if needed. If I wanted to start with a regexp instead, I would just add a space at the beginning to start the second query. For some commands, I would probably want to reverse that order (or maybe I could just do query 1 regexp, query 2 initialism, query 3 regexp instead of having different defaults for different commands; I'd have to play around with it).

In another example, maybe I realize I didn't type the initialism correctly (e.g. included too many letters). Instead of deleting the query, I could change it to fuzzy by using . (or whatever) as a separator after it.

I guess this would require adding more commands for configuring separators. It's a little more complicated, but it would be completely optional, and it seems like it could be worth it to me. Most of the benefit for me would come from having different default matching types for different queries, so I think having different separators for different query types would be the less often useful part. Something similar (but not quite as flexible) could be achieved by just initially hitting the separator key enough times to get to the query type you want. I guess you could add even more rules to also be able to pick a specific query type beforehand, like you could specify two separators in a row... but that makes things even more complicated. I think it's probably enough to be able to change the query type afterward to handle the "oops, I actually need fuzzy" case. And even then, if it was the first query, you could just go to the beginning of the line and add the necessary number of separators to get it to be in the fuzzy query position. So maybe using separators to specify the query type isn't useful enough to justify the extra complexity.

What do you think? Do you have any other ideas?

[clemera]

I had another idea: Why not allow orderless-component-matching-styles to also be a function. This would be very intuitive and would already cover most use cases I think. Most people probably want either a global dispatcher or component wise dispatchers.

If orderless-style-dispatchers are set they are used for component wise matching styles, otherwise fallback to orderless-component-matching-styles which can also be function (of the discussed form). This way there would be only one user option to configure the global matching style and for everything more sophisticated one can use orderless-style-dispatchers or orderless-pattern-compiler directly. What do you think?

[oantolin]

I think I like this idea of folding in this "whole input" dispatcher into the default pattern compiler (and leave modifying the pattern compiler for people doing something really different), as a way to computing the default matching styles for all components, overridable on individual componentes by the component style dispatchers.

And as you say, one way to do this is to let orderless-component-matching-styles be a function. But what I don't like about this, is that writing that function feels pretty different from using orderless-style-dispatchers, even though it is the same kind of thing: compute some styles and possibly a new string. The "pipeline" logic of trying things one at a time until you figure out how to transform the input and what styles to use is handled for you in the component case but not in the whole input case and is probably pretty similar. I think maybe you were right about adding a variable all along, but it should be a dispatch list.

We could rename orderless-style-dispatchers to orderless-component-style-dispatchers and introduce another dispatcher list called orderless-global-style-dispatchers. I'd introduce a function orderless-dispatch to run lists of dispatchers and then the overall logic of the default patter compiler would be:

1. Find the default matching styles for components, this is given by (orderless-dispatch orderless-global-style-dispatchers input) or, if that gives an empty style list, fallback on orderless-component-matching-styles.

2. For each component compute the styles with (orderless-dispatch orderless-component-style-dispatchers componet index total) falling back to the default computed in the previous step.

How does that sound?

Most people probably want either a global dispatcher or component wise dispatchers.

I don't know, I think of the global dispatcher as a "computed orderless-component-matching-styles". Whether your list of default styles is constant or computed, you might still want to be able to override it for individual components. In the example global dispatcher you wrote, wouldn't it be nice in the trailing-comma-flex-matching case to flag some specific component for a literal match?

This is why I'm in favor of just adding the global dispatching to the default pattern compiler (which I plan to rename from orderless--component-regexps to orderless-default-pattern-compiler). And given that, it would feel weird to me if component-wise dispatching were done by running a list of dispatchers, but global dispatching was instead done with a single function (that in most cases will probably have a logic similar to trying a sequence of cases in order, i.e., running a list of dispatchers).

[clemera]

How does that sound?

Pretty cool but wouldn't that make the component dispatchers dependent on the result of the global dispatchers? If you have to coordinate those two I could imagine that would get complicated quickly.

In the example global dispatcher you wrote, wouldn't it be nice in the trailing-comma-flex-matching case to flag some specific component for a literal match?

I'm not sure I understand what you mean here, could you give a more detailed example how that would look like?

it would feel weird to me if component-wise dispatching were done by running a list of dispatchers, but global dispatching was instead done with a single function (that in most cases will probably have a logic similar to trying a sequence of cases in order, i.e., running a list of dispatchers).

As I said above I'm concerned the fallback of the component dispatchers to other dynamic dispatchers would get complicated quickly. I think the fallback of component wise dispatchers should probably be static.

[oantolin]

Pretty cool but wouldn't that make the component dispatchers dependent on the result of the global dispatchers? If you have to coordinate those two I could imagine that would get complicated quickly.

I don't think it would be hard to coordinate. Don't think of the global dispatchers as computing the matching styles for the components, think of them as computing the default matching styles for the components, that is, think of them as providing "a computed orderless-component-matching-styles".

The component dispatchers try to specify matching styles for each component (and possibly transform the component in the process). If they don't produce an actual list of styles to use there is a default that gets used instead. Currently that default is orderless-component-matching-styles (or 'orderless-regexp if that variable is nil). I'm just proposing to change the default from "the static orderless-component-matching-styles" to "orderless-component-matching-styles, whether static or computed".

oantolin: In the example global dispatcher you wrote, wouldn't it be nice in the trailing-comma-flex-matching case to flag some specific component for a literal match?

clemera: I'm not sure I understand what you mean here, could you give a more detailed example how that would look like?

Say you used a "#" prefix for literal matches. Then abc def ghi, would mean flex matching for each of abc, def, and ghi; and abc #def ghi, would mean flex matching for abc and ghi (because they get the dynamically computed default matching style of "flex"), and literal matching for def (because it overrides the (computed) default).

As I said above I'm concerned the fallback of the component dispatchers to other dynamic dispatchers would get complicated quickly. I think the fallback of component wise dispatchers should be static.

Does thinking of global dispatch as just providing a computed value for orderless-component-matching-styles help or do you still think this will confuse people?

[oantolin]

Maybe it is complicated but at least it's a non-branching hierarchy:

1. there are matching styles computed by the component dispatchers,
2. default matching styles computed by the global dispatchers,
3. default default matching styles given by orderless-component-matching-styles, and
4. a default default default matching style decreed to be orderless-regexp.

[oantolin]

OK, the last proposal I made is implemented in commit 6c05cf8! The vast majority of the effort was writing docstrings. :P

Here's how an example configuration looks. I'll do @clemera's previous global dispatch example, with the addition of component hash literal overrides. So we want:

• regexp matching if there is more than component,
• but a trailing comma overrides that and uses flex matching for all components (removing the comma),
• otherwise (single component, no trailing comma) use strict leading initialism,
• allow individual components to match literally with a # prefix.

That would be:

(defun flex-if-comma (pattern &rest _)
  (when (string-suffix-p "," pattern)
    `(orderless-flex . ,(substring pattern 0 -1))))

(defun literal-if-hashtag (pattern &rest _)
  (when (string-prefix-p "#" pattern)
    `(orderless-literal . ,(substring pattern 1))))

(defun regexp-if-many-components (pattern &rest _)
  (when (string-match-p " " pattern)
    'orderless-regexp))

(setq orderless-component-dispatchers '(literal-if-hashtag))
(setq orderless-global-dispatchers '(flex-if-comma regexp-if-many-components))
(setq orderless-component-matching-styles 'orderless-strict-leading-initialism)

I like that the little function can be used for global or component dispatching by placing them in the appropriate list, they feel more reusable this way (of course, if they do use the component index then they can only be used for component dispatching). I think this looks a little neater than @clemera's "all global dispatching goes into a single function", and it looks analogous to the component dispatching.

Let me say upfront that I don't mind at all scrapping this whole global dispatchers thing if we think it's too complicated, it's just a proposal.

Also, if we do go with this global dispatchers list, we can consider removing the "total number of components" argument from the component dispatchers: let things that depend on the number of components be global.

So, please let me know what you think, @clemera and @noctuid. (Also, @noctuid note that if you test this latest commit to the dispatch branch, orderless-style-dispatchers has been renamed to orderless-component-dispatchers.)

Also, should orderless-component-matching-styles be renamed to something like orderless-default-matching-styles (with an alias for backwards compatibility)?

I'd rather not push something to master without at least us three agreeing it is reasonable.

[clemera]

Thanks for your elaboration and excellent work! I will test this and try to give some feedback the next few days. I was coming from the viewpoint to use either a global dispatcher or component wise dispatchers so I needed some time to adjust to the idea to combine the two. Also I haven't checked the code in detail at all yet.

One example of what I'm thinking currently: What if I need to change the matching style of all components? For example if I use the above as default and now I start typing:

"foo+ ^bar $baz"

Now I want to switch to literal search or regex search for the whole input. Or maybe I want to configure the global way via a prefix like "r " for regex searching and "l " for literal searching and start my input with that to mean all following components should be matched this way.

[oantolin]

What if I need to change the matching style of all components?

Oh, boy! This sounds like it could be useful. I guess maybe sometimes you want component dispatcher's to override global ones, and sometimes the other way around.

We can always just say: "if you want global overriding local, the default compiler won't help you: write your own!".

But say we do want to include that case in the default pattern compiler? With what interface? I'm out of ideas at the moment. I do feel that, if we are to solve this problem, instead of adding yet another things on top of what we have, it would be better to step back and think.

Let's test what we have and in parallel think about whether there is a nice redesign that adressess this too. (I'm not overly attached to the current code and would drop it in a heartbeat if we came up with a simpler design that still covered all the cases that have been brought up.)

[clemera]

Maybe the solution can be simple: Adding something like orderless-overriding-dispatchers which override all others. Similar to emacs keymaps: overriding-terminal-local-map (set-transient-map), local-map, global-map :smile:

[oantolin]

I wanted to avoid that, but maybe it's not so bad. I guess it would be orderless-overriding-global-dispatchers, because the component dispatcher already override the global dispatchers.

[clemera]

After some sleep I think I would prefer a list like orderless-overriding-matching-styles one could set. This way one could define keybindings to rotate throuch the styles toggle them or set them explicitly (similar to isearch where you do this with `M-r´ and other bindings.

[oantolin]

There are so many options here: toggle individual styles on or off (this could be nice with a hydra), cycle between a list of preset style-lists, select preset style-lists via some key (either a keymap or hydra would do here, probably), etc. So I think this belongs to the realm of personal configuration. I don't think any particular way of interactively changing styles should be blessed in the package. (In fact I really want to remove orderless-temporarily-change-separator, which I added impulsively and doubt anyone uses.)

Maybe we could just have a wiki page on the topic of interactively changing matching styles? We could put snippets like this one there:

(define-key minibuffer-local-completion-map (kbd "M-m")
  (let ((presets '((?r preset:regexp orderless-regexp)
                   (?i preset:initials orderless-strict-initialism)
                   (?f preset:flex orderless-literal orderless-flex))))
    (cl-loop with map = (make-sparse-keymap)
             for (key fn-name . styles) in presets
             do
             (fset fn-name
                   (lambda ()
                     (interactive)
                     (setq orderless-component-matching-styles styles)))
             (put fn-name 'function-documentation
                  (format "Use matching styles: %s" styles))
             (define-key map (vector key) fn-name)
             finally (return map))))

[clemera]

I'm not proposing to add commands which actually do this, I agree that those should be user configuration. But this time I'm pretty sure this is not a niche request ;)

Ivy which has started long before orderless existed had this feature very early on. As you might know it implements its own matchers and filtering styles thus they had to decide how to support selecting a matching method at some point, too. AFAIK ivy does not support such a cool feature as a component dispatcher which is very nice to have but I think the simpler more obvious (I really need to get rid of that phrase ;) users probably want a way to set the completion style is globally by a key binding or with some kind of menu (which frameworks can provide or users can write for themselves).

I think providing something like orderless-overriding-global-dispatchers in addition to the above might still make sense for the people who want full control of the matching styles via their input. I can't think of any other options one might want and adding these two last considerations would feel complete to me. But of course you have to decide.

[clemera]

One note to your code example: Using orderless-component-matching-styles would mean this would get overridden by the dispatchers, if you consider adding such a feature I think it needs to override all the dispatchers. It basically should mean: Don't care what I have as input, I really want this or that matching style.

[oantolin]

One note to your code example [...]

You can add

(setq orderless-component-dispatches nil
      orderless-global-dspatchers    nil)

to that code example if you wish.

I think I'm a little confused about what this orderless-overriding-global-dispatchers variable is for. Let me ask you several questions about it to clarify:

1. Is the idea that

(setq orderless-overriding-global-dispatchers some-list)

has the same effect as the following?

(setq orderless-component-matching-styles some-list
      orderless-component-dispatches      nil
      orderless-global-dspatchers         nil)

2. If so, why the extra variable? Why not just set the other 3 as above? Is the idea that you can set this new variable just for a little while and then when you do

(setq orderless-overriding-global-dispatchers nil)

you haven't clobbered the previous values of orderless-component-matching-styles, orderless-component-dispatches, and orderless-global-dspatchers?

So instead of a new variable people could just save and restore the 3 existing ones?

3. You say Ivy has a similar mechanism? Could you provide more detail about that? Even just a variable name would be useful.

[oantolin]

Oh, when you talked about Ivy, did you mean the ivy-preferred-re-builders variable? (But you'd suggest having it without the corresponding ivy-rotate-preferred-builders function, right?)

[clemera]

You can add ... to that code example if you wish.

But than your dispatchers are gone. You probably don't want that because you set them up in your user configuration and wouldn't like to have to restore them just because you wanted the current input have another matching. Think of it like in isearch where you can switch to different matching styles as well. You wouldn't want any variables you set in your config be overriden just because you switched to regexp search one time.

The global orderless-overriding-global-dispatcher is less important to me than an orderless-overriding-matching-styles I talked about above but I though it would make sense to have it, too. Just in case the user wants for example define prefixes to force specific input styles for the whole following input. For example you could have a prefix like "l " to get:

"l some$ stuff+ that matches literally" 

hence the effect wouldn't be the same as setting orderless-component-dispatches and orderless-global-dispatchers to nil because you would loose the value of that variables. Personally I wouldn't use this because as I said I would prefer using something like orderless-overriding-matching-styles.

Could you provide more detail about that?

I haven't used it for a while but you have various toggle functions for example ivy-toggle-fuzzy and so on. Using those you can switch to another matching and switch back to the previous one you had. And as you found there are also other mechanism like ivy-rotate-preferred-builders.

But you'd suggest having it without the corresponding ivy-rotate-preferred-builders function, right?

From glancing at the code of ivy I'm basically suggesting having something like ivy--regex-function which is used to determine the matching style for current input. This would be the equivalent to orderless-overriding-matching-styles I was talking about. A global variable which when set defines the current matching style and is meant to be dynamically changed. All the other variables are basically statically bound to the values in your personal config.

[clemera]

Having orderless-overriding-matching-styles being a private variable would also be possible like ivy--regex-function is. But I think because it would be intended to be set by commands the user writes I don't think it should be private.

[clemera]

To avoid confusion (or to add more;): I know ivy--regex-function is similar to the pattern compiler. I only meant it equivalent in terms of how it operates for the UI: Setting it via commands changes the matching style. The last resort in orderless is the pattern compiler with which you could also switch to different matching styles per commands but the whole point is convenience, setting available styles instead having to deal with the regexps yourself any additional logic.

[oantolin]

I'm sorry, I'm still confused. :(

Can you say as explicitly as possible why you'd want orderless-overriding-matching-styles?

You say you want it to behave like ivy--regex-function in the sense that setting it via commands changes the matching style.

1. In the current master branch, orderless-component-matching-styles behaves that way, does that mean that in the current master a orderless-overriding-matching-styles variable is unnecessary?

2. In the dispatch branch, the trio of variables orderless-component-matching-styles, orderless-component-dispatchers and orderless-global-dspatchers, taken as a unit, have that property: setting them changes the matching style used!. Why would we need an overriding variable? Why not just set those 3 variables to whatever you want?

3. If you set orderless-overriding-matching-styles it would remain in effect until you set it to nil again. I see two cases:

• If the user is OK with changes to orderless-overriding-matching-styles staying in effect after the current completion session, then why have an extra variable? Just set the trio of existing variables.

• If the user wants to confine the effect of setting orderless-overriding-matching-styles to the current completion session, then some code has to set it to nil afterwards. If the user has to write code to set to nil again, why not just write code to restore the trio of existing variables?

I swear I'm not playing dumb: I genuinely am struggling to understand. Can you say very, very explicitly why setting this one overriding variable would be better that setting the trio?

[oantolin]

Is the whole point that you want do deal with only one variable? That can easily be arranged: we can use a single variable for the configuration.

We could replace the current

(setq orderless-component-matching-styles '(A ...)
      orderless-global-dispatchers        '(B ...)
      orderless-component-dispatchers     '(C ...))

with something like:

(setq orderless-component-matching-styles
      '(:default-styles A ...
        :global-dispatchers B ...
        :component-dispatchers C ...))

(I'd make the :default-styles keyword optional for backward compatibility.)

[clemera]

I swear I'm not playing dumb: I genuinely am struggling to understand.

No worries ;) I swear I also try my best to explain myself.

If the user wants to confine the effect of setting orderless-overriding-matching-styles to the current completion session, then some code has to set it to nil afterwards. If the user has to write code to set to nil again, why not just write code to restore the trio of existing variables?

I also have thought of that, my idea was that this could be done by orderless in minibuffer-exit-hook.

I'm going to bed now so I will respond to the other points in more detail tomorrow. Just as a quick note about what the whole point is: I'm aiming to be able to use all of the futures discussed so far in conjunction. So maybe I want to have component dispatchers AND the ability to toggle the matching with a key. All of that without much ceremony regardless of which combinations I choose (I'm currently unsure what combo I will use personally, maybe I won't use component wise dispatching at all but what to do if I change my mind in the future EDIT: Or more importantly what if more users want this). As I said I have to sleep now, I will elaborate more tomorrow.

[noctuid]

In fact I really want to remove orderless-temporarily-change-separator, which I added impulsively and doubt anyone uses.

I actually did use it once (because there's no way to escape spaces). Am fine with putting it in my personal config though.

[clemera]

Good morning ;)

does that mean that in the current master a orderless-overriding-matching-styles variable is unnecessary?

No, as you mentioned the variable needs to be reset which I think could be done when entering/exiting the minibuffer. I think of orderless-overriding-matching-styles as the the default matching style configured in users init file in which case it shouldn't be reset by orderless.

Why would we need an overriding variable? Why not just set those 3 variables to whatever you want?

Because this is more work and would need to be done manually and you would also have to take care to restore the variables each time after you reset them. I'm trying to find a convenient way to support a way which supports all scenarios without much intervention by the user. Maybe that can be achieved another way, too and I just don't see it currently.

If the user is OK with changes to orderless-overriding-matching-styles staying in effect after the current completion session

No because as I mentioned above I think of them as the default matching styles and thus they shouldn't change permanently.

If the user wants to confine the effect of setting orderless-overriding-matching-styles to the current completion session, then some code has to set it to nil afterwards. If the user has to write code to set to nil again, why not just write code to restore the trio of existing variables?

As in my last comment yesterday, I think orderless might could do that in minibuffer exit hook. EDIT: Doing it in setup hook would mean one couldn't start a temporary overriding style using minibuffer-with-setup-hook which could be nice, so I think the exit hook should be used.

We could replace the current

(setq orderless-component-matching-styles '(A ...)
      orderless-global-dispatchers        '(B ...)
      orderless-component-dispatchers     '(C ...))

with something like:

(setq orderless-component-matching-styles
      '(:default-styles A ...
        :global-dispatchers B ...
        :component-dispatchers C ...))

I really like that idea in combination of the orderless-overriding-matching-styles we are talking about. That way there would only be two variables, to configure the styles: The default and component wise methods combined into one (with the handy property that you can just add more keywords in the future without introducing new variables). And then a second variable to transiently set the styles via a key binding (thinking about it maybe something like orderless-transient-overriding-matching-styles would be a better name).

Writing about this it just comes to mind that if you decide to go that route, the transient styles variable could also use the same format as the default one so they could be treated the same in the implementation and the user could switch the component wise dispatchers via key binding as well (although I'm not sure one would really like to do that but you never now).

[oantolin]

Thanks for your patience! So the point is that you'd want to save the user the trouble of resetting the matching styles. Fair enough! I think I'm very close to being on board now. How about just transient, that is, orderless-transient-matching-styles?

And for consistency I could add a transient separator variable. (That way people writing their own substitute for the interactive separator changing function I marked obsolete, will have a much easier time writing it.)

Does the single variable combining default matching styles and global and component dispatchers sound OK to you @noctuid ? (Described here.)

[oantolin]

Writing about this it just comes to mind that if you decide to go that route, the transient styles variable could also use the same format as the default one

Yes, I assumed it would, @clemera.

[clemera]

How about just transient, that is, orderless-transient-matching-styles?

Sounds better to me, too.

Yes, I assumed it would

I hadn't though about using the new format you proposed with it before, but it just felt into place when writing/thinking about it :)

[clemera]

If you decide to implement this, I think it would make sense to add a global minor mode. The mode could set completion-styles and add the hook to exit-minibuffer-hook.

[clemera]

Another nice feature this approach would cover is that it allows to easily setup different default matching styles per command using something like:

(add-hook 'minibuffer-setup-hook
          (defun my-maybe-set-orderless-style-for-command ()
            (setq
             orderless-transient-matching-styles
             (cdr-safe (assq this-command my-orderless-cmd-style-alist)))))

[oantolin]

If you decide to implement this, I think it would make sense to add a global minor mode. The mode could set completion-styles and add the hook to exit-minibuffer-hook.

Maybe we can just provide the function to add to the hook and document it's use in docstring and in the README. I think that providing copy-pasteable configuration is enough for this, it doesn't require a whole minor mode that only resets the transient configuration.

[clemera]

Okay, the suggestion of the global minor mode was wrong anyway (the keymap for the transient commands would need to be active in the minibuffer only).

[oantolin]

OK, done! The new single variable configuration + transients system are finished (9cebf2f). (Again the vast majority of the work was writing docstrings and updating the README :P).

But I'm suddenly indecisve: with the inclusion of this transient configuration and of the total argument for component dispatchers, I'm not sure we need global dispatchers anymore. And even if we do want the global dispatchers, should the component dispatchers override them or the other way around? (Currently it's as discussed above: component dispatcher override global dispatchers.)

I hope you guys can take a look and test this, @clemera, @noctuid.

[clemera]

Very cool, thank you! I will look at this in more detail on the weekend. I think I would indeed like to have the global dispatchers have precedence, as this would allow to force some style for the whole input (as we discussed previously when talking about overriding disptachers). The fallback of the component dispatchers to the static default styles should be enough, I guess. At least I currently can't think of an example where you would need to have it the other way around with the current approach. I think the current design is very nice.

[oantolin]

How about removing global dispatchers altogether (keeping the component dispatchers)? I feel their uses are better served by transients.

Like a typical use for global dispatchers is to be able to say, "forgo all special processing for this query, just match all components as literals". I think a little command that sets the transient matching style to literal matching is a good choice for this.

(I'm more fond of removing code than adding it.)

[clemera]

After some thinking I think they can be removed in their current form, too. I'm not trying to annoy you but I have another idea in mind which would introduce another variable instead (related to the changing the whole input matching by input). Are you still open to discussion or are you tired about it? I would be okay with that, too.

EDIT: Oh I think the idea could also be implemented without another variable, maybe that appeals to you more ;)

[oantolin]

I'm still game. :) What was your idea?

[clemera]

Great! Actually I came to the conclusion that the idea I had wouldn't work out. But I still think the current design is not fully complete yet. As I see it there are the following possible common scenarios what users might want:

1. Use some set of default styles for all commands.
2. Use some default styles but also use kind of a query language to control the styles via input.
3. Start specific commands with a custom set of styles.
4. Switch to a set of styles by pressing some key binding.

And here is how they can solve those with the current design (assuming the code with :global-dispatchers got already removed):

1. Just set orderless-component-matching-styles to a list of styles they want.
2. Set orderless-component-matching-styles to a list of styles and setup :component-dispatchers to have specific components use a different matching style.
3. Use orderless-transient-matching-styles in minibuffer-setup-hook.
4. Write commands which set orderless-transient-matching-styles and bind them to keys in minibuffer-setup-hook.

This already sound pretty good but I still have some questions:

What if the user sets up 3. and doesn't specify :component-dispatchers? Should the query language he usually uses be totally ignored? Maybe he just wants to setup a different list of styles but still be able to use his query language defined by his default :component-dispatchers. Should there be a way to tell the :component-dispatchers of orderless-transient-matching-styles to fallback to the :component-dispatchers of orderless-component-matching-styles?

And lastly, wouldn't it be nice if the query language would be more complete? By that I mean that the ability to use custom prefixes to force styles which are applied to the whole input, is still appealing to me. You are right that you can just use the mechanism in 4. and setup key bindings but I think it could be very convenient to allow patterns like the following:

"l literal$ match+"
"r regex$,+match"
...

Because using this technique you don't need to press a key binding first. Usually you know what kind of match you want when you start typing. So to regex match you just start with "r " and input your regex. There is no modifier key to press and you can just start typing. Also you wouldn't have to think about a smart key binding which has no conflicts with other keys.

This way of writing your input would be intentional by the user and setting up transient styles can be automatic (via minibuffer-setup-hook) I think because of this, this technique would need to have the highest precedence overall.

[clemera]

To expand on:

What if the user sets up 3. and doesn't specify :component-dispatchers? Should the query language he usually uses be totally ignored? Maybe he just wants to setup a different list of styles but still be able to use his query language defined by his default :component-dispatchers. Should there be a way to tell the :component-dispatchers of orderless-transient-matching-styles to fallback to the :component-dispatchers of orderless-component-matching-styles?

After all the :component-dispatchers and default style list maybe shouldn't be packed into one variable again. Then you can easily switch the default styles with transient styles and keep oderless-component-dispatchers separate. I don't think wanting to have transient component dispatchers would be very common.

[clemera]

This way of writing your input would be intentional by the user and setting up transient styles can be automatic (via minibuffer-setup-hook) I think because of this, this technique would need to have the highest precedence overall.

Wait, now that we have transient styles this could be solved by setting the transient style in component dispatchers (assuming :component-dispatchers keyword is replaced by a variable again as mentioned above). With that in mind I think you can ignore this point, I would be happy with that.

[oantolin]

I'm still thinking, but here are some preliminary comments:

About global dispatchers for stuff like l literal$ match+: it seems to me that here global dispatch is really only saving you repeating the type of match in every component. For example, I've been trying an = suffix for literal matches, so instead of l literal$ match+, I'd use literal$= match+=. I guess the savings can be larger if you have a bunch of components, but I rarely do. Particularly for literal matching, it's seems to be pretty rare to need it, I'm usually matching letters which need no regexp escaping.

Maybe we'd need to compile a list of possible uses of global dispatcher to see how important they are, but global matchers like this l, which just say "use this matching style for all components" seem like a trivial shortcut over repeating a component dispatcher on each component (with the advantage that you can change style per component, for example if you wanted match+ literally but literal$ as regexp, your global l isn't useful, but I can go literal$ match+=).

What if the user sets up 3. and doesn't specify :component-dispatchers? Should the query language he usually uses be totally ignored? Maybe he just wants to setup a different list of styles but still be able to use his query language defined by his default :component-dispatchers.

That can be easily done by simplying copying the :component-dispatchers to the transient variable:

(setq orderless-transient-matching-styles
      (append '(some transient styles)
              (memq :component-dispatchers 
                    orderless-component-matching-styles)))

Of course, a helper function can be provided, so that this can be reduced to

(setq orderless-transient-matching-styles
      (orderless-with-dispatchers '(some transient styles)))

(The name can be debated.)

Wait, now that we have transient styles this could be solved by setting the transient style in component dispatchers (assuming :component-dispatchers keyword is replaced by a variable again as mentioned above).

That wouldn't work with the current implementation: the dispatchers run after checking whether there are transient setting or not. (This could be changed, but I'd rather not.)

[oantolin]

I also just realized your l literal$ match+ syntax can be achieved with component dispatchers, no need for global ones or transients:

(defvar my-default-styles '(orderless-regexp orderless-initialism))

(defun all-literal-if-first-component-is-l (pattern index _total)
  (when (and (= index 0) (string= pattern "l"))
    (setq my-default-styles 'orderless-literal)
    '(orderless-regexp . "")))

(defun my-default-dispatcher (_pattern _index _total)
  my-default-styles)

(setq orderless-component-matching-styles
      '(:component-dispatchers
        all-literal-if-first-component-is-l
        my-default-dispatcher))

[oantolin]

Obviously that's a trick and we may not want that to the official solution... specially since it has a bug! Once you backspace over the initial l it should restore the default styles. :P

Here's a corrected version:

(defvar my-default-styles '(orderless-regexp orderless-initialism))

(defun all-literal-if-first-component-is-l (pattern index _total)
  (if (and (= index 0) (string= pattern "l"))
      (progn
        (setq my-default-styles 'orderless-literal)
        '(orderless-regexp . ""))
    (setq my-default-styles '(orderless-regexp orderless-initialism))
    nil))

(defun my-default-dispatcher (_pattern _index _total)
  my-default-styles)

(setq orderless-component-matching-styles
      '(:component-dispatchers
        all-literal-if-first-component-is-l
        my-default-dispatcher))

I'm not saying this nice or pretty, I'm just pointing out that component dispatchers can implement the l literal$ match+ syntax. In practice I wouldn't do this, I'd just repeat the literal marker for each component.

[clemera]

That can be easily done by simplying copying the :component-dispatchers to the transient variable:

Seems you would prefer to specify the fallback each time instead of changing the type of the variable. Is it because you don't want an additional variable? Do you think transient component dispatchers could be useful?

I'm not saying this nice or pretty, I'm just pointing out that component dispatchers can implement the l literal$ match+ syntax

You would also have to additionally adjust it to use the regular :compnent-dispatchers one might want to use when not using the global prefix (just want to point out that this is getting complicated and not user friendly).

[oantolin]

Do you think transient component dispatchers could be useful?

I'm starting to think they might, here's a proposal:

• We think of matching styles + component dispatchers as together providing a "query language", as you said above.

• We think of global dispatchers as preprocessing the input and specifying what query language (so matching styles + component dispatchers) to use.

• Transients provide an query language overriding everything.

So the variable orderless-component-matching-styles would hold the default query language: just the styles and the component dispatchers. The global dispatchers would be moved to a new variable ordeless-preprocessors (I think for clarity if we go with this proposal we should rename component dispatchers to just dispatchers and rename global dispatchers to preprocessors).

The logic in general would be:

• If orderless-transient-matching-styles is non-nil, then it wins and specifies the matching styles and component dispatchers.

• Otherwise, the orderless-processors are run to determine a possibly modified input string and which styles+dispatchers to use (styles+dispatchers defaults to orderless-component-matching-styles).

Seems you would prefer to specify the fallback each time

Yes, because I feel that you may or may not want it (for example you might want l ... to match every component literally except as modified by the component dispatchers, but you might want L ... to match every component literally with no exceptions). Maybe it's best to be explicit about when you do want to use the component dispatchers and when you don't, even if it can be a little wordier in the case you do want the fallback.

How does this sound? I think the key difference is that I'm now thinking in terms of the styles+component-dispatchers unit.

[oantolin]

I'm still not convinced that the global dispatchers or preprocessors are really worth the extra cognitive load.

Currently global dispatchers say "use these style for every single component, except as overridden by the component dispatchers" and they can be replaced by repeating a component dispatcher (re abc def ghi --> re:abc re:def re:ghi), gaining flexibility in the process (and picking single character markers in the components, it's really not much wordier, it's more like re abc def ghi vs ;abc ;def ;ghi, something like that).

The preprocessors of the above proposal are slightly different, because with them you aren't forced to use the set of component dispatchers in orderless-component-matching-styles, you can specify to use a different list or none at all. But again, they can probably be replaced by just typing slightly more in the invidual components.

I like the preprocessor version slightly better than the current global dispatches, but either way? Do we really need them? What's something cool you can do with them that's really awkward without them?

[clemera]

Your proposal sounds interesting but also too complicated for interactive use. As you said there is cognitive load and I think usually you would want to keep the same component dispatchers in all contexts. That is why I would favour a single variable which defines them. I wouldn't be able to remember different dispatchers as defined by code. Having one query language for components would be enough to remember for me already. I think when the query language currently in use is dependent on what orderless-processors would determine to use, I would probably have no idea what behaviour my input currently has.

[oantolin]

I think usually you would want to keep the same component dispatchers in all contexts.

I'm inclined to agree, but be aware that if you always use the component dispatchers, then you cannot have something like l literal$ match+ ... to match all components literally with no exceptions. The closest you could do is l literal$ match+ ... meaning to match all components literally except as modified by the component dispatchers. Are you OK with that?

[oantolin]

I think my current favorite proposal is the current dispatchers branch but removing global dispatchers, which is exactly the same as what you get from my latest proposal by removing the "preprocessors".

And if global dispatchers cannot change the component dispatchers or at least just turn them off, I don't really see the value in them.

[clemera]

I think my current favorite proposal is the current dispatchers branch but removing global dispatchers, which is exactly the same as what you get from my latest proposal by removing the "preprocessors".

Does this mean you would like to keep the transient component dispatchers? I really believe that you risk serious brain damage when component dispatchers can change on the fly ;)

[clemera]

Are you OK with that?

I'm willing to give up on the idea of input prefixes which define the whole input matching for now. Maybe better ideas how to combine that with what we have so far come up later.