Introduce a selectrum-config file?

[clemera]

Currently we collect configurations in the wiki. They can be easily overlooked there and also are inconvenient to track and update in user configurations. Maybe we could introduce a selectrum-config file which would take care of integrations with built-ins and other packages automatically if they are present. Loading those configurations could be controlled by a user option (non-nil by default) WDYT?

[okamsn]

Do you mean similar to how smartparens suggests users do (require 'smartparens-config)?

I would appreciate that, especially for packages that default to ido instead of the normal completing-read (e.g., Projectile in #112).

Other inclusions could be for disabling sorting, such as for flyspell-correct, which adds special candidates to the end of the list. But maybe that one is more preference than need.

[clemera]

Do you mean similar to how smartparens suggests users do (require 'smartparens-config)?

I don't use smartparens but probably that is a similar approach. We could put something like

(when selectrum-auto-config 
    (require 'selectrum-config)
    (run-hooks 'selectrum-auto-config-hook))

in the mode setup for selectrum. By default the option selectrum-auto-config would be t and the hook would contain several functions to take care of different setups. Having a hook should make it easy for users to adjust the default configurations if needed.

Other inclusions could be for disabling sorting, such as for flyspell-correct, which adds special candidates to the end of the list.

I think in this case it would be better to request that flyspell-correct provides the correct meta data for the completion table so sorting is disabled (default completion also does sort based on history so if you don't want sorting it's good practice to stop it like this).

[raxod502]

This seems reasonable, with the caveats that:

• code included in Selectrum proper should be held to a higher standard of quality than code on the wiki
• not necessarily all of the configurations in the wiki should be included, only the ones that make sense in all use cases (or perhaps they can be put inside minor modes?)

It just requires a bit of thought to decide what to include and in what way.

See also https://github.com/raxod502/straight.el/blob/master/straight-x.el, which contains user-contributed extensions that are not yet mature enough to be included in the main package.

[clemera]

I have worked on a draft in #174 where we can discuss things, further. In general I think it could be a nice convenience for Selectrum users.

[minad]

@clemera Wouldn't it make more sense to create another package which does this or distribute an example configuration in another form? I would find it nice if we have a configuration package where we could also toggle between the different components, like selectrum/icomplete, prescient/orderless etc. Even if people have good reasons to prefer selectrum over icomplete, icomplete is still a reference implementation.

[clemera]

I like your idea but it seems different in scope and Selectrum and icomplete aren't equivalent in features so you can not switch between them without doing a lot of checking how to translate a feature set from on to the other. My goal is to make it easy for Selectrum users to setup Selectrum for packages that are known to work well with Selectrum.

[minad]

@clemera Yes they are not completely equivalent. But another proposal I thought about making was to perform this radical selectrum api reduction, e.g. remove most public api except for the configuration interface you are designing, add support for dynamic candidates via completing-read etc. But try to stay within the confines of completing-read. The only big blocker which I am seeing is this dynamic candidate issue. I wonder which packages/functionality works only with selectrum? Generally I think selectrum works better than icomplete, faster, more intuitive etc (I am using it myself mostly), but I wonder which features there are which are not somehow also present in icomplete? For example we had this narrowing issue, which worked better in selectrum before but with the narrowing prefix+keymap-hack it also works nicely with the standard api.

[clemera]

another proposal I thought about making was to perform this radical selectrum api reduction, e.g. remove most public api except for the configuration interface you are designing, add support for dynamic candidates via completing-read etc. But try to stay within the confines of completing-read.

Thanks to your great work lately this might really be an option now. We can have all the great features without breaking the API!

I wonder which packages/functionality works only with selectrum? Generally I think selectrum works better than icomplete, faster, more intuitive etc (I am using it myself mostly), but I wonder which features there are which are not somehow also present in icomplete? For example we had this narrowing issue, which worked better in selectrum before but with the narrowing prefix+keymap-hack it also works nicely with the standard api.

You are right I think mostly UI things now and customization options like default sorting method, display actions and such. But still the configuration is Selectrum specific (what variables need to be set and so on). If we change things/add new features we want to have direct control over the auto setup offered to users so I would like to have them in Selectrum.

[minad]

Thanks to your great work lately this might really be an option now. We can have all the great features without breaking the API!

Thank you! It wouldn't have worked out with your and @oantolin's help! Thank you both :)

You are right I think mostly UI things now and customization options like default sorting method, display actions and such. But still the configuration is Selectrum specific (what variables need to be set and so on). If we change things/add new features we want to have direct control over the auto setup offered to users so I would like to have them in Selectrum.

I think Selectrum should still have certain UI tweaks available as configuration options - maybe with the potential to upstream them conceptually in form of metadata or via some other mechanism. As I said the only thing which is really missing is the dynamic candidate support via the completing-read api - something like this is needed in any case where you want to dynamically generate candidates while the user types. But at the same time this should be offered such that via the selectrum-specific optimizations the reloading behavior can be tweaked.

Regarding the config discussion here - I agree with you that it makes sense to have some curated config available which works particularly well for selectrum. But from the architectural pov, I really do like this concept of having these pluggable components. This is what really makes the difference to the monoliths and ensures reusability.

[clemera]

As I said the only thing which is really missing is the dynamic candidate support via the completing-read api

@minad Note that there are various problems with dynamic candidates sets when using the default api, too. I investigated and described this a bit more in #114. In fact I don't think you can replicate the dynamic function feature we currently have in Selectrum with the default API without ugly hacks (overriding the completion style like I did in #114).

[minad]

Note that there are various problems with dynamic candidates sets when using the default api, too. I investigated and described this a bit more in #114. In fact I don't think you can replicate the dynamic function feature we currently have in Selectrum with the default API without ugly hacks (overriding the completion style like I did in #114).

Sorry, I didn't read yet through the details you wrote and I am not yet very familiar with how these dynamic tables could work. Right now I only worked with the static api and pushed this as far as I could ;) I hope we can get something simple working with dynamic candidate sets - the most basic possible thing, but I have to check what specific use case you have been looking at in the #114 discussion.

[minad]

@clemera Shall we reconsider this now? I think no such configuration should be provided by Selectrum. Consult and Embark both auto-configure when combined with Selectrum or another completion system. Marginalia works out of the box. Orderless works out of the box like a usual completion style. I think we solved everything in a principled way. The only odd one out is Prescient unfortunately - we discussed this multiple times already. But this does not justify adding a preconfiguration here which would be problematic as a melpa package due dependencies. It is also not a good idea to add another layer in between the packages and the init.el use-package configuration.

@bdarcus Where do you see the need for better configuration? What do you want to see simplified? An example Selectrum configuration consists of a single line, (selectrum-mode), same with Marginalia. Embark needs an action keybinding but this must be configured anyways since we do not know the user preference. The only long configuration is Consult, but this is also mostly keybindings, which I do not consider to be complicated. If one would preconfigure Consult it would almost certainly clash with user preferences, which can differ greatly - evil or not and so on.

[clemera]

@minad I agree, one thing we could do is to link more prominently from the usage section to the config examples on the wiki or additionally include a minimal example config using all those packages with some of the typical adjustments one might want to make.

[bdarcus]

Where do you see the need for better configuration? What do you want to see simplified? An example Selectrum configuration consists of a single line, (selectrum-mode), same with Marginalia.

My understanding was the inspiration for this (and more so #174) was making it easier for users to config the suite of packages.

I don't know exactly what that should look like, but I could imagine different what we might call recipes of mixes of packages.

Whether that's worthwhile at this point is for you all to decide, but I'm sure you've all observed (I've seen it noted in discussions) that people used to integrated solutions like ivy and helm get confused by these decoupled packages.

I imagine that has to carry through to how they approach installation and configuration.

But this from @clemera can also work:

... one thing we could do is to link more prominently from the usage section to the config examples on the wiki or additionally include a minimal example config using all those packages with some of the typical adjustments one might want to make.

[minad]

@bdarcus I have about 50 packages configured in my configuration or more. In Emacs distributions you have a lot more. I have to configure each of them. There is nothing special about our packages here which justifies a mix package.

The value proposition of Selectrum, Icomplete, Embark, Consult, Marginalia, Orderless, Prescient is that you can combine them freely according to your preference. You can as well use Icomplete-vertical instead of Selectrum or decide between Prescient and Orderless. There are even more complementary packages to to be combined. By providing some mix package this exact value is lost.

I am not against having an extra mix package outside the selectrum package/repository, since this is where it would actually belong. But in the end this will turn out to be an opinionated configuration/distributions which will not satisfy all needs. Many users are happy to have the flexibility down to the details. And for users interested in preconfigured environments there can be support in Emacs distributions.

I don't know exactly what that should look like, but I could imagine different what we might call recipes of mixes of packages.

This is not concrete enough. I don't see what is being proposed. The packages are flexible, they can be configured using the (more or less) standard Emacs facilities (use-package and customization). And they even auto-configure where necessary in order to ease integration and in order to prevent configuration mistakes.

You may remember that there had been a separate consult-selectrum package which had to be installed separately via MELPA. This led to confusion without added value and I had many bug reports due to this package being missing. By automatically installing the consult-selectrum integration with consult this has been fixed.

However technically this is not free. The integration package has to provide function/defvar predeclarations and will break more easily when Selectrum is changed since the warnings by the bytecode compiler have effectively been disabled. See https://github.com/minad/consult/blob/c5d74b191fb4ee70509e2592fa16a9398f96977b/consult-selectrum.el#L27. Note that consult-selectrum does not specify Selectrum as dependency.

For this reason the currently accepted best practices regarding MELPA packages do recommend against magic auto configuration. I also had some discussions about that with the MELPA maintainers. Only in very special cases auto configuration is advised. And one should consider this carefully as we have done for the integrations.

But now if one would provide a larger mix package it would auto-configure on every aspect since one would not want to include all components as hard dependencies. As argued this is problematic. Alternatively the mix package could specify hard dependencies but then the very modularity aspect is lost.

people used to integrated solutions like ivy and helm get confused by these decoupled packages

I don't think it is actually confusing. The decoupling of the components makes the parts easier to understand and more robust.

For example when I am testing Consult I can run it against Emacs default completion, Selectrum and Icomplete. When I look at full blown Ivy/Helm configurations they are not shorter than a configuration of the various aforementioned packages.

Note that one also has to distinguish arguments being made on a technical basis vs arguments being made only with the goal to attack some core value of the other packages. The topic of completion systems has for some reasons been a heated one. The Selectrum readme could benefit from some adjustments to make it more objective and less adversarial. I already made a few adjustments myself at some point (#290), before I got actually more technically involved with the project. Note also issue #457 by @raxod502.

But this from @clemera can also work:

I agree with the statement by @clemera too and this is where I actually see something actionable. The wiki and readme should include detailed example configurations. This is what I am trying to do. The configuration examples in the READMEs of the Consult and Marginalia packages, which I maintain/co-maintain, are made such that you can copy them verbatim to a plain config and they will work as is. If people come with suggestions for improvements or if we come up with new insights, the documentation will be adjusted accordingly. I believe that to be the right approach. I can not make a definitive statement regarding the other packages readmes in particular Selectrum, but I think the same applies to them.

As a package user, if you have improvement suggestions it is probably best to make them on that level.

[bdarcus]

I agree with the statement by @clemera too and this is where I actually see something actionable. The wiki and readme should include detailed example configurations. This is what I am trying to do. The configuration examples in the READMEs of the Consult and Marginalia packages, which I maintain/co-maintain, are made such that you can copy them verbatim to a plain config and they will work as is.

Yes, and they do that well.

If people come with suggestions for improvements or if we come up with new insights, the documentation will be adjusted accordingly. I believe that to be the right approach. I have to check the other package READMEs too, I think the same applies to them.

As a package user, if you have improvement suggestions it is probably best to make them on that level.

Fair enough.

To be clear, I'm thinking about the user who wants to experiment with selectrum, but haven't yet grokked the basic value or design here.

I actually have a couple of super simple suggestions that might help.

First, in the complementary extensions section of the docs, there is this sentence:

The wiki also contains configuration tips for external packages.

... with a link to this wiki page:

https://github.com/raxod502/selectrum/wiki/Additional-Configuration

But the examples at the top of the page actually don't address the external packages, and that section is only later on that page.

So the suggestion is either:

1. create a new page for external config and move that section there, and have that link point to that, or ...
2. have the link directly to that section

I think there then might be room for improvement on the content itself, but I need to think on that more.

Maybe a start would be a sentence or two to begin that section suggesting new users install embark, marginalia and consult (maybe orderless also)?

Prescient is already recommended, and the example config included, on the main docs, so that's not necessary there, although you could refer back to it.

Config examples for them might be appropriate, though I can see you might worry about overlap.

[minad]

Yes, the readme+wiki of Selectrum should be improved. I agree. But this has also grown over the last year. Some of the information may even be outdated.

To be clear, I'm thinking about the user who wants to experiment with selectrum, but haven't yet grokked the basic value or design here.

One thing I have considered is to add like a quick start section to the selectrum/consult readmes.

1. Package-install selectrum
2. package-install consult
3. Enable selectrum-mode
4. Ensure that some appropriate completion style is used, e.g. substring. Emacs basic is too basic.
5. Run some completing read command, find-file, switch-to-buffer or some consult command consult-line, consult-buffer

[clemera]

@minad

The topic of completion systems has for some reasons been a heated one.

I guess this partly because of the work people invested learning about the options and all the quirks of the framework they currently use. Often they carefully picked that framework considering the alternatives at the time so if something new comes along that can be frustrating because they are challenged to reconsider and even might come to the conclusion that it might be a better fit for them which would mean a lot of adjustments and work again loosing most of the old ones. Although I generally like to have choices for software myself it can definitely be a burden, too.

The wiki and readme should include detailed example configurations.

I started working on this in #472 where we can discuss this further.

@bdarcus

...create a new page for external config and move that section there, and have that link point to that, or have the link directly to that section

Good suggestion, we should also reorganize things on the wiki a bit.

[minad]

Close this? I think we covered it by our test scripts and the readme improvements.