+include: ~/OrgFiles/armin/org-macros.setup

+OPTIONS: h:1 num:nil toc:nil d:nil

+TITLE: consult-omni - a powerful versatile omni search inside Emacs

+AUTHOR: Armin Darvish

+LANGUAGE: en

+html:

+html:

• About =consult-omni=

consult-omni is a package for getting search results from one or several custom sources (web search engines, AI assistants, elfeed database, org notes, local files, desktop applications, mail servers, ...) directly in Emacs minibuffer. It is a successor of [[https://github.com/armindarvish/consult-web][consult-web]], with expanded features and functionalities.

consult-omni provides wrappers and macros around [[https://github.com/minad/consult][consult]], to make it easier for users to get results from different sources and combine local and web sources in an omni-style search. In other words, consult-omni enables getting consult-style multi-source or dynamically completed results in minibuffer for a wide range of sources including Emacs functions/packages (e.g. Emacs buffers, org files, elfeed,...), command-line programs (grep, find, gh, ...), or web search engines (Google, Brave, Bing, ...).

consult-omni can be an open-source free alternative to other omni-search tools such as Alfred or MacOS spotlight. It provides a range of default sources as examples, but the main idea here is to remain agnostic of the source and provide the toolset for the users to define their own sources/workflows (a.k.a plugins).

Here is the mandatory screenshot:

+ATTR_ORG: :width 800px

+ATTR_LATEX: :width 800px

+ATTR_HTML: :width 800px

[[https://github.com/armindarvish/consult-web/blob/screenshots/screenshots/consult-omni.gif]]

In the screenshot above, I use a key shortcut to open an Emacs frame and call =consult-omni= (similar to calling MacOS spotlight) and search for the term “emacs”, then move around and look at previews from different sources including application, gptel, Brave, Google, YouTube, elfeed, mu4e, ....

Here is another screenshot demonstrating using consult-omni as an application launcher:

+ATTR_ORG: :width 800px

+ATTR_LATEX: :width 800px

+ATTR_HTML: :width 800px

[[https://github.com/armindarvish/consult-web/blob/screenshots/screenshots/consult-omni-launcher.gif]]

• Usage and Features For explanation of features and comparison to some other packages, you can watch my YouTube videos, both for consult-web and for consult-omni below:

1. (video 1) in-depth tutorial for consult-omni: [[https://www.youtube.com/watch?v=wNH2E7iT__c][https://www.youtube.com/watch?v=wNH2E7iT__c]]
2. (video 2) in-depth tutorial of consult-web: https://www.youtube.com/watch?v=7pDfyqBZwvo

** Interactive Commands *** single source commands For each source, you may have static or dynamic commands. Static commands query the use for an input and then fetch the results. Dynamic commands, do dynamic completion as the user types (show results as user is typing). Dynamic commands feel a bit more intuitive and modern in 2024, but on the other hand have the disadvantage of sending the query to the servers multiple times especially if you type slowly! Depending on the service provider and the API model, you may want to avoid hitting the server too frequently (for example for services that you pay per query), therefore for certain services a static command might be a better choice than the dynamic command. Using the macro =consult-omni-define-source=, you can chose to create static, dynamic or both by passing =nil=, =t=, or ='both= to the keyword =:static=. Here is an example from the source code, for creating both static and dynamic commands for Brave search:

+begin_src emacs-lisp

(consult-omni-define-source "Brave" :narrow-char ?b :type 'dynamic :require-match t :face 'consult-omni-engine-title-face :request #'consult-omni--brave-fetch-results :preview-key consult-omni-preview-key :search-hist 'consult-omni--search-history :select-hist 'consult-omni--selection-history :enabled (lambda () (bound-and-true-p consult-omni-brave-api-key)) :group #'consult-omni--group-function :sort t :static 'both)

+end_src

Here is a screenshot of STATIC interactive command for Wikipedia:

+ATTR_ORG: :width 800px

+ATTR_LATEX: :width 800px

+ATTR_HTML: :width 800px

[[https://github.com/armindarvish/consult-web/blob/screenshots/screenshots/consult-omni-wikipedia-static.gif]]

Here is a screenshot of DYNAMIC interactive command for Wikipedia:

+ATTR_ORG: :width 800px

+ATTR_LATEX: :width 800px

+ATTR_HTML: :width 800px

[[https://github.com/armindarvish/consult-web/blob/screenshots/screenshots/consult-omni-wikipedia.gif]]

*** multi source commands By default consult-omni provides two main multi-source interactive commands.

1. =consult-omni-multi=: This is an interactive DYNAMIC command that uses multiple sources, as defined by =consult-omni-multi-sources=, and shows the results in minibuffer completion with dynamic completion (results are fetched as the user types). Here is a screenshot:

+ATTR_ORG: :width 800px

+ATTR_LATEX: :width 800px

+ATTR_HTML: :width 800px

[[https://github.com/armindarvish/consult-web/blob/screenshots/screenshots/consult-omni-multi.gif]]

2. =consult-omni-multi-static=: This is an interactive STATIC command that uses multiple sources, as defined by =consult-omni-multi-sources=, and shows the results in minibuffer completion. Note that consult-omni-multi-static does not provide dynamic completion but some might find using this more intuitive for narrowing down the results. The user provides one search term, and once the results are retrieved, typing in the minibuffer will narrow down the candidates.

Note that unlike the predecessor consult-web, consult-omni does not provide other multi-source interactive commands (e.g. scholar search) anymore. It's now up to the user to define such commands for their custom workflow. For example one user may define separate interactive commands to query web sources, v.s. local sources, while another user may want to define interactive commands for knowledge sources (academic references, notes, ...) v.s. utilities (apps, buffers, files,...). Examples for how to define new interactive commands are provided below in the advanced config [[id:DFBCF797-44D6-4265-B737-77916700C43D][here]].

** Dynamic Completion: Passing Arguments and Narrow Down

Arguments can be passed to the dynamic interactive commands and further narrowing down the results can be done using a syntax similar to the “Perl splitting” style in [[https://github.com/minad/consult?tab=readme-ov-file#asynchronous-search][consult asynchronous search]].

For narrowing down the results you need adding =#= (or another character defined in =consult-async-split-style=) after the search query. For example typing the following in the minibuffer:

+begin_example

emacs web search#github

+end_example

First searches for “emacs web search”, and then uses “github” for narrow down.

Furthermore, arguments can be passed to dynamic commands using similar syntax as =consult-grep=, too. For example typing the following in the minibuffer:

+begin_example

how to browse a url in emacs -- --model gpt-3.5-turbo

+end_example

passes =gpt-3.5-turbo= as the value for the keyword argument =:model= to the back-end functions of all the sources that fetch results. If any of those sources accept the keyword argument =:model=, the value =gpt-3.5-turbo= gets passed to them. For this reason it is recommended to always use functions that accept any keyword arguments (a.k.a. add =&allow-other-keys=) to avoid errors when non-existing keywords are passed to them.

instead of using =--= , you can also use a keyword with colon =:=. The following would be similar to the example above:

+begin_example

how to browse a url in emacs -- :model gpt-3.5-turbo

+end_example

** Understanding different Type of Sources

When using consult, we deal with different types of sources, either elisp functions (e.g. =buffer-list=, =re-search-forward=) or command-line programs (e.g. =grep=, =notmuch=, =gh=, =mu=, ...). They return a list of candidates, which in turn are passed to =completing--read=. While consult provides a way to combine multiple sources with =consult--multi=, the ability to combine sources of different types (command-line program with elisp for example) is limited. In consult-omni we solve this by creating some wrappers and machinery around built-in consult functions and by having each source declare its type. This type tells consult-omni how to collect candidates from that source and combine with other sources. The type can be ='sync=, ='dynamic=, or ='async=.

• ='sync=: This is a synchronous source, meaning that when consult-omni calls the request function to collect candidates from this source, it calls it synchronously (blocks the Emacs process) and waits for a returned value (expecting a list of candidates or nil). This is for example suitable for sources that simply run an elisp function and return a list quickly like =buffer-list=. There is almost no gain in calling this function asynchronously because the overhead for async control (timers and watchers,...) is probably going to cost more time than just calling the function directly and synchronously.

• ='dynamic=: Dynamic sources are those that would benefit or require calling the collecting elisp function asynchronously. For example, if collecting items from a source requires to send an HTTP request to a server and waiting for the response, then a dynamic type should be used, so that Emacs process is not blocked while waiting for the response. In this case the request function for the source (the function that returns the items) should take a callback function that will be called when the response arrives. This is for example is suitable for getting response from web search APIs like Google's, Brave's or OpenAI's APIs.

• ='async=: asynchronous sources, are those that use an external command line program and therefore require to create process, get the output of the command line program and parse it to collect candidates. This is for example suitable for getting candidates from =grep=, or =notmuch=, ....

consult-omni machinery allows combining multiple sources of different types in one command by combining sync and async calls/processes so that the user can efficiently get search results from various sources in almost real-time (no waiting needed, the candidate are shown as they arrive).

** Embark Actions You can load the default embark actions by;

+begin_src emacs-lisp

(require 'consult-omni-embark)

+end_src

The default actions allow you to open the links in the default or alternate browser and also to copy or insert, title and/or URL of the links. Other embark actions can be defined per your own specific work flow.

See the YouTube video on consult-web for an example, here: [[https://youtu.be/7pDfyqBZwvo?t=4962]].

** Other Important Features *** Minimal Code Base Without doc-strings and whitespaces the code is less than 1000 lines and it only depends on [[https://github.com/minad/consult][consult]] and built-in url-retrieve.

*** Modular You can only load the parts you need. For example if all you need is an autosuggestion utility similar to =helm-google-autosuggest=, then you can use minimal config like this:

+begin_src emacs-lisp

(use-package consult-omni :straight (consult-omni :type git :host github :repo "armindarvish/consult-omni" :branch "main" :files (:defaults "sources/*.el")) :after consult :config ;; Load Sources Core code (require 'consult-omni-sources) ;; Load Embark Actions (require 'consult-omni-embark) ;; Only load brave-auto-suggest source (require 'consult-omni-brave-autosuggest) ;;; Set your shorthand favorite interactive command (setq consult-omni-default-interactive-command #'consult-omni-brave-autosuggest))

+end_src

Note that every module (a.k.a. every source) adds an extra 100-200 lines of code. This also means to add a new source, you only need to write a short piece of code following those examples!

*** Customizable and Extendable Lots of customization options both for sources and also for running actions on the results. New sources can be added as you wish with different format, different actions,...

*** Power User Capabilities Dynamic collection allows for complex workflows on the fly. Change query parameters on the fly by passing arguments. Select a random set of results ad-hoc using embark and run embark actions on them. This allows batch processing as well. For example to add a long list of results to an org-mode note for later review (as shown in this YouTube video: [[https://youtu.be/7pDfyqBZwvo?t=4774]]).

• Getting Started

** Precautions Before you start, make sure you understand three points:

1. Important Note 1: This is work in progress in its early stage and bugs and issues are very much expected.
2. Important Note 2: You should consider the general risks of using Emacs to browse the web. By default, all codes are trusted inside Emacs and browsers are naturally the target of many attacks. Therefore, it is important to be aware of the risks and be intentional about what links you open (or do not open) inside Emacs. By default, consult-omni would only be opening web pages (or calling APIs of) the sources (e.g. search engines, ...) and not any other websites. It's up to the user then to decide how she or he wants to open the links and chose their own risk tolerance. consult-omni provides customization variables for different actions (e.g. opening links in external browser v.s. in Emacs), so make sure you know how to set everything up.
3. Important Note 3: The functions provided in =consult-omni-sources=, provide a basic demonstration for integrating different services (such as search providers), however since each service comes with its own terms and conditions (that may change over time and vary from location to location), it is difficult to provide all-encompassing solutions and maintain them over time. consult-omni is agnostic of how you connect and integrate other services in your setup (because neither consult-omni nor Emacs collect any information of the users or their usage), and therefore ultimately only you the user are responsible for setting up everything correctly and understand consequences of the usage (e.g. costs of using paid APIs) and ensure to stay within the bounds of relevant laws and regulations for your use case (i.e. follow software user agreements, etc.). Therefore, it is important for you to read and understand how to use each service, and also understand what happens under the hood when you integrate the service with consult-omni. I try my best to provide documentation here as well as on the [[https://github.com/armindarvish/consult-omni/wiki][wiki pages]], and will try to help when possible but before you proceed understand that you do everything at your own risk.

* Installation If you want an example config see [[https://github.com/armindarvish/consult-omni?tab=readme-ov-file#drop-in-example-config][Drop-in Example Config*]]. Here is some detailed explanation;

*** Requirements In order to use consult-omni, you need Emacs >28.0 (I have not tested earlier versions) and you need [[https://github.com/minad/consult][consult]]. While this is the only requirements, I suggest you read the rest of this README. I do recommend some other packages and useful configurations for different settings. Some of those extra packages and settings can improve your experience of consult-omni, therefore you may want to consider installing them as well. For example, combining consult with other packages such as [[https://github.com/minad/vertico][vertico]], [[https://github.com/oantolin/orderless][orderless]], and [[https://github.com/oantolin/embark][embark]] can improve the functionality as well as user-experience.

*** Installing consult-omni Package consult-omni is not currently on [[https://elpa.gnu.org/packages/consult.html][ELPA]] or [[https://melpa.org/#/consult][MELPA]]. Therefore, you need to install it using an alternative non-standard package manager such as [[https://github.com/radian-software/straight.el][straight.el]], [[https://github.com/progfolio/elpaca][elpaca]], ... or use manual installation.

**** straight.el To install consult-omni with straight.el you can use the following command. Make sure you load consult-omni after loading consult (e.g. =require 'consult=).

+begin_src emacs-lisp

(straight-use-package '(consult-omni :type git :host github :repo "armindarvish/consult-omni" :files (:defaults "sources/*.el")))

+end_src

or if you use =use-package= macro with straight, you can do:

+begin_src emacs-lisp

(use-package consult-omni :straight (consult-omni :type git :host github :repo "armindarvish/consult-omni" :files (:defaults "sources/*.el")) :after consult)

+end_src

You can also fork this repository and use your own repo.

**** manual installation Clone this repo and make sure the files are on your load path, as described on [[https://www.emacswiki.org/emacs/LoadPath][EmacsWiki]].

Make sure you load consult (e.g. =require 'consult=) before you load consult-omni.

** Adding Search Sources * Loading Sources ** load all default sources at once You can search ALL the default sources by loading the provided =consult-omni-sources= module, and then calling the function =consult-omni-sources-load-modules=:

+begin_src emacs-lisp

(require 'consult-omni-sources) (consult-omni-sources-load-modules)

+end_src

This provides sources for some popular services, and adds a long list of interactive commands (dynamic search, static search or both depending on the source). Over time I hope to add more services, hopefully by contribution from the community as well. Note that these are also good examples for you to learn how to add your own sources or tweak the current ones for your specific use-cases.

**** load multiple sources but not all You can also load multiple sources (but not all) by setting the list =consult-omni-sources-modules-to-load= and then loading calling =consult-omni-sources-load-modules=:

+begin_src emacs-lisp

(require 'consult-omni-sources) (setq consult-omni-sources-modules-to-load '(consult-omni-google consult-omni-wikipedia)) (consult-omni-sources-load-modules)

+end_src

This limits the sources that consult-omni loads to ONLY those defined in =consult-omni-sources-modules-to-load=.

**** load a single source Alternatively you can load a single source by just requiring the corresponding file. For example for Wikipedia, you can do:

+begin_src emacs-lisp

(require 'consult-omni-sources) (require 'consult-omni-wikipedia)

+end_src

This would add interactive commands only for searching Wikipedia (e.g. =consult-omni-wikipedia= and/or =consult-omni-wikipedia-static=).

*** List of Sources Provided by Default Here is a list of current sources. For details on how to set and use each source, refer to the [[https://github.com/armindarvish/consult-omni/wiki][wiki pages]] (under construction now). You can also watch the YouTube Tutorial video here: [[https://www.youtube.com/watch?v=wNH2E7iT__c][https://www.youtube.com/watch?v=wNH2E7iT__c]]

|--------------------------+----------------------------------------------| | Source | Category | |--------------------------+----------------------------------------------| | Apps | Desktop Applications | | Bing | Web Search Engine | | Brave AutoSuggest | Web Word Completion/AutoSuggest | | Brave | Web Search Engine | | Browser History | Web Browser History | | consult buffer sources | Emacs Buffers, Files, Bookmarks, ... | | calc | Emacs Calculator | | chatGPT | Simple AI prompts | | consult-notes | Local Notes | | Dictionary | Emacs Built-in Dictionary | | doiorg | Academic Reference | | DuckDuckGo (Limited API) | Search Suggestions | | elfeed | Feeds (RSS, videos,...) | | fd | Search Local Files with /fd/ Command | | find | Search Local Files with /find/ Command | | GitHub | Search GitHub Repos with /gh/ Command | | Google | Web Search Engine | | Google AutoSuggest | Web Word Completion/AutoSuggest | | grep | Search Local Files with /grep/ Command | | git-grep | Search Local Git Files with /grep/ Command | | ripgrep | Search Local Files with /rg/ Command | | ripgrep-all | Search Local Files with /rga/ Command | | gptel | AI Assistant | | Invidious | Online Videos (YouTube) | | buffers text search | Search Text in Emacs Buffers | | locate | Search Local Files with /local/ Command | | man | Search Manual Pages with /man/ Command | | mdfind | Search Local Files with OSX /mdfind/ Command | | mu4e | Search Emails | | Notes Search | Search Local Note Files | | notmuch | Search Emails | | Numi | Calculator with numi-cli/ command from [[https://github.com/nikolaeu/numi][Numi]]) | | Org agenda | Search Org Agenda Items | | PubMed | Academic Reference | | Scopus | Academic Reference | | StackOverflow | Community Forum | | Wikipedia | Encyclopedia | | YouTube | Online Videos (YouTube) | |--------------------------+----------------------------------------------|

** Configuration consult-omni is built with the idea that the user should be able to customize everything based on their use-case, therefore the user is very much expected to learn how to configure this package. Therefore, I recommend you read through this section and understand how to configure the package according to your needs and for your specific use-case. In addition to these settings, some sources have their own customization variables, etc. which is discussed in the [[https://github.com/armindarvish/consult-omni/wiki][wiki pages]] (under construction now) and inn this YouTube Tutorial: https://www.youtube.com/watch?v=wNH2E7iT__c.

*** Customization Variables The following customizable variables are provided:

**** =consult-omni-sources-modules-to-load= List of modules to load. By setting this you before calling =consult-omni-sources-load-modules=, you can limit the modules (sources) that are loaded and can avoid errors due to missing requirements for certain sources. For example for just loading Wikipedia:

+begin_src emacs-lisp

(setq consult-omni-sources-modules-to-load (list 'consult-omni-wikipedia))

+end_src

**** =consult-omni-default-browse-function= Default browse function for opening URLs. This can be set to external browser function by:

+begin_src emacs-lisp

(setq consult-omni-default-browse-function 'browse-url)

+end_src

**** =consult-omni-alternate-browse-function= Secondary browse function for opening URLs. This can for example be set to eww or some other browsers for quick access to an alternative browser with embark actions.

+begin_src emacs-lisp

(setq consult-omni-alternate-browse-function 'eww-browse-url)

+end_src

**** =consult-omni-default-preview-function= Default function to use for previewing links. This can for example be set to [[https://www.gnu.org/software/emacs/manual/html_mono/eww.html][eww]]:

+begin_src emacs-lisp

(setq consult-omni-default-preview-function #'eww-browse-url)

+end_src

or [[https://www.gnu.org/software/emacs/manual/html_node/emacs/Embedded-WebKit-Widgets.html][xwidegt-webkit]]:

+begin_src emacs-lisp

(setq consult-omni-default-preview-function #'xwidget-webkit-browse-url)

+end_src

**** =consult-omni-show-preview= This turns previews on/off for all consult-omni sources. It is recommended to set this to =t= and use =preview-key= to control previews per source.

**** =consult-omni-preview-key= This is the default preview key. A good choice might be "C-o".

+begin_src emacs-lisp

(setq consult-omni-preview-key "C-o")

+end_src

**** =consult-omni-default-format-candidate= This variable a stores a function that is called for formatting the candidates (on any source that does not have an explicit formatting function). consult-omni provides two example functions:

• =consult-omni--simple-format-candidate= shows the tile without marginalia annotation in minibuffer
• (Default) =consult-omni--highlight-format-candidate= shows candidate with marginalia annotation and highlights the search term as well.

*** =consult-omni-default-count= By default consult-omni retrieves only up to* this many results per source. It is recommended to keep this to a low number (e.g. 5 to 10) to keep the performance fast. The default is set to 5 because nowadays for most everyday use-cases, you probably won't need more than the top 5 results.

+begin_src emacs-lisp

(setq consult-omni-default-count "5")

+end_src

Keep in mind that with dynamic commands of consult-omni, you can always increase the number by passing arguments to =-n=, =--count=, =:n=, or =:count= (for example by typing =search term -- -n 30=, you can retrieve up to 30 results.

*** =consult-omni-default-page= This is similar to going to page 2, 3,...,N on a classic search result page. If this is set to N, first (N count/page) results are skipped and the next page of results are shown. It is recommended to keep this as default, 0, to see the top results.

Keep in mind that with dynamic commands of consult-omni, you can always change the page by passing values to arguments =-p=, =--page=, =:p=, or =:page= (for example by typing =search term -- -p 2=, you can get page 2.

=consult-omni-default-timeout= Maximum time in seconds before =consult-omni= gives up in fetching results from a single source in STATIC commands. =consult-omni-url-use-queue= A boolean, whether to use =url-queue= or not. =consult-omni-url-queue-parallel-processes= Number of =url-queue= processes that runs in parallel. =consult-omni-url-queue-timeout= Maximum time in seconds that a single =url-queue= job can live before giving up. =consult-omni-log-buffer-name= Buffer name for logging information, which is mostly useful for debugging. =consult-omni-log-level= This variable controls how consult-omni logs information or displays errors. It can be either =nil=, ='info=, or ='debug=, in the order of logging/showing more details.

**** =consult-omni-group-by=

This is the field that is used to group the results. By default, results are grouped by the name of the source. But this can be changed to achieve various grouping behavior. If this is set to =nil= , the results are not grouped. It can also be set to other properties of the candidates (e.g. =:url=, =:title=, =:domain=, ...)

+begin_src emacs-lisp

(setq consult-omni-group-by :source)

+end_src

**** =consult-omni-multi-sources= This is a list of source name strings (e.g. ='("Google", "Wikipedia", "chatGPT")=) that will be used in the commands =consult-omni-multi=, and =consult-omni-multi-static= (see above for description).

=consult-omni-highlight-matches-in-minibuffer= Whether consult-omni highlights matches of the search term in the minibuffer candidates. This is useful to highlight the relevance of the search results. =consult-omni-highlight-matches-in-file= Whether consult-omni highlights matches of the search term in files (e.g. in previews or when opening files). This is useful for finding matches of the search term aftr opening target files.

**** =consult-omni-default-interactive-command= This is a convenient feature to bind your favorite consult-omni interactive command to the command called =consult-omni=, so it is easier to remember and find when you call =M-x=. You can bind this to one of the interactive commands (such as =consult-omni-multi= or =consult-omni-multi-static=) or you can bind it to a single-source command (e.g. =consult-omni-google=, =consult-omni-google-static=, =consult-omni-google-wikipedia=, ...) or alternatively you can define your own custom command (single or multi-source) and use that instead.

**** =consult-omni-http-retrieve-backend= This variable controls which back-end is used by consult-omni for HTTP requests. consult-omni supports 3 different back-ends:

• (Default) built-in =url-retrieve=
• =request= (see [[https://github.com/tkf/emacs-request][emacs-request]])
• =plz= (see [[https://github.com/alphapapa/plz.el][plz.el]])

For example to change the back-end to =plz=, you can do:

+begin_src emacs-lisp

(setq consult-omni-http-retrieve-backend 'plz)

+end_src

**** =consult-omni-default-autosuggest-command= Default autosuggest command. consult-omni provides two examples with =consult-omni-dynamic-brave-autosuggest= and =consult-omni-dynamic-google-autosuggest=, but you can also define other custom autosuggest commands from other sources (e.g. google, Wikipedia, ...)

+begin_src emacs-lisp

(setq consult-omni-default-autosuggest-command #'consult-omni-dynamic-brave-autosuggest)

+end_src

**** =consult-omni-dynamic-input-debounce= In dynamic commands, the dynamic collection process is started only when there has not been new input for =consult-omni-dynamic-input-debounce= seconds. If you type slow or think you need time to think for what you want to search, you may want to increase this number, so you don't run searches prematurely, especially if you want to avoid running premature search terms on paid services. By default this inherits from consult's built-in input-debounce value, which is 0.5. Personally I find that a bit too fast for consult-omni because I do not want consult-omni to send a query to paid OpenAI API while I am still typing my query so I slow it down to 0.8 - 1s.

+begin_src emacs-lisp

(setq consult-omni-dynamic-input-debounce 0.8)

+end_src

**** =consult-omni-dynamic-input-throttle= In dynamic commands, the dynamic collection process is started only every =consult-omni-dynamic-input-throttle= seconds. If you use API services that have limited number of queries per second, you may want to increase this number to avoid getting errors. I set this to 2x my input-debounce value:

+begin_src emacs-lisp

(setq consult-omni-dynamic-input-throttle 1.6)

+end_src

**** =consult-omni-dynamic-refresh-delay= In dynamic commands, the completion UI is only updated every =consult-omni-dynamic-refresh-delay= seconds. You probably want to run this as fast as =consult-omni-dynamic-input-debounce=.

+begin_src emacs-lisp

(setq consult-omni-dynamic-input-throttle 0.8)

+end_src

** Drop in Example configs

*** Minimal config for testing the Package

+begin_src emacs-lisp

(use-package consult-omni :straight (consult-omni :type git :host github :repo "armindarvish/consult-omni" :branch "main" :files (:defaults "sources/*.el")) :after consult :custom ;; General settings that apply to all sources (consult-omni-show-preview t) ;;; show previews (consult-omni-preview-key "C-o") ;;; set the preview key to C-o :config ;; Load Sources Core code (require 'consult-omni-sources) ;; Load Embark Actions (require 'consult-omni-embark)

;; Only load wikipedia source (setq consult-omni-sources-modules-to-load (list 'consult-omni-wikipedia)) (consult-omni-sources-load-modules)

;;; Set your shorthand favorite interactive command (setq consult-omni-default-interactive-command #'consult-omni-wikipedia))

+end_src

*** A reasonable config with mostly default Settings

+begin_src emacs-lisp

(use-package consult-omni :straight (consult-omni :type git :host github :repo "armindarvish/consult-omni" :branch "main" :files (:defaults "sources/*.el")) :after consult :custom ;; General settings that apply to all sources (consult-omni-show-preview t) ;;; show previews (consult-omni-preview-key "C-o") ;;; set the preview key to C-o :config ;; Load Sources Core code (require 'consult-omni-sources) ;; Load Embark Actions (require 'consult-omni-embark)

;; Either load all source modules or a selected list

;;; Select a list of modules you want to aload, otherwise all sources all laoded ; (setq consult-omni-sources-modules-to-load (list 'consult-omni-wkipedia 'consult-omni-notes)) (consult-omni-sources-load-modules) ;;; set multiple sources for consult-omni-multi command. Change these lists as needed for different interactive commands. Keep in mind that each source has to be a key in `consult-omni-sources-alist'. (setq consult-omni-multi-sources '("calc" ;; "File" ;; "Buffer" ;; "Bookmark" "Apps" ;; "gptel" "Brave" "Dictionary" ;; "Google" "Wikipedia" "elfeed" ;; "mu4e" ;; "buffers text search" "Notes Search" "Org Agenda" "GitHub" ;; "YouTube" "Invidious"))

;; Per source customization

;;; Set API KEYs. It is recommended to use a function that returns the string for better security. (setq consult-omni-google-customsearch-key "YOUR-GOOGLE-API-KEY-OR-FUNCTION") (setq consult-omni-google-customsearch-cx "YOUR-GOOGLE-CX-NUMBER-OR-FUNCTION") (setq consult-omni-brave-api-key "YOUR-BRAVE-API-KEY-OR-FUNCTION") (setq consult-omni-stackexchange-api-key "YOUR-STACKEXCHANGE-API-KEY-OR-FUNCTION") (setq consult-omni-pubmed-api-key "YOUR-PUBMED-API-KEY-OR-FUNCTION") (setq consult-omni-openai-api-key "YOUR-OPENAI-API-KEY-OR-FUNCTION")

;;; Pick you favorite autosuggest command. (setq consult-omni-default-autosuggest-command #'consult-omni-dynamic-brave-autosuggest) ;;or any other autosuggest source you define

;;; Set your shorthand favorite interactive command (setq consult-omni-default-interactive-command #'consult-omni-multi))

+end_src

*** Advanced config with customized interactive commands, etc. :PROPERTIES: :ID: DFBCF797-44D6-4265-B737-77916700C43D :END: Here is a drop-in config snippet that puts everything mentioned above together. Read the sections above for more details.

+begin_src emacs-lisp

(use-package consult-omni :straight (consult-omni :type git :host github :repo "armindarvish/consult-omni" :branch "main" :files (:defaults "sources/*.el")) :after consult :custom

;;; General settings that apply to all sources

(consult-omni-show-preview t) ;;; show previews (consult-omni-preview-key "C-o") ;;; set the preview key to C-o (consult-omni-highlight-matches-in-minibuffer t) ;;; highlight matches in minibuffer (consult-omni-highlight-matches-in-file t) ;;; highlight matches in files (consult-omni-default-count 5) ;;; set default count (consult-omni-default-page 0) ;;; set the default page (default is 0 for the first page)

;; optionally change the consult-omni debounce, throttle and delay. ;; Adjust these (e.g. increase to avoid hiting a source (e.g. an API) too frequently) (consult-omni-dynamic-input-debounce 0.8) (consult-omni-dynamic-input-throttle 1.6) (consult-omni-dynamic-refresh-delay 0.8)

;; Optionally set backend for http request (either 'url, 'request, or 'plz) (consult-omni-http-retrieve-backend 'plz)

:config

;;; Load Sources Core code (require 'consult-omni-sources)

;;; Load Embark Actions (require 'consult-omni-embark)

;;; Either load all source modules or a selected list ;; Select a list of modules you want to aload, otherwise all sources all laoded ; (setq consult-omni-sources-modules-to-load (list 'consult-omni-wkipedia 'consult-omni-notes)) (consult-omni-sources-load-modules)

;; set multiple sources for consult-omni-multi command. Change these lists as needed for different interactive commands. Keep in mind that each source has to be a key in `consult-omni-sources-alist'. (setq consult-omni-multi-sources '("calc" ;; "File" ;; "Buffer" ;; "Bookmark" "Apps" ;; "gptel" "Brave" "Dictionary" ;; "Google" "Wikipedia" "elfeed" ;; "mu4e" ;; "buffers text search" "Notes Search" "Org Agenda" "GitHub" ;; "YouTube" "Invidious"))

;;; Per source customization

;; Set API KEYs. It is recommended to use a function that returns the string for better security. (setq consult-omni-google-customsearch-key "YOUR-GOOGLE-API-KEY-OR-FUNCTION") (setq consult-omni-google-customsearch-cx "YOUR-GOOGLE-CX-NUMBER-OR-FUNCTION") (setq consult-omni-brave-api-key "YOUR-BRAVE-API-KEY-OR-FUNCTION") (setq consult-omni-stackexchange-api-key "YOUR-STACKEXCHANGE-API-KEY-OR-FUNCTION") (setq consult-omni-pubmed-api-key "YOUR-PUBMED-API-KEY-OR-FUNCTION") (setq consult-omni-openai-api-key "YOUR-OPENAI-API-KEY-OR-FUNCTION") ;; add more keys as needed here.

;; gptel settings (setq consult-omni-gptel-cand-title #'consult-omni--gptel-make-title-short-answer)

;; default terminal (setq consult-omni-embark-default-term #'vterm)

;; default video player (setq consult-omni-embark-video-default-player #'mpv-play-url)

;; pretty prompt for launcher (setq consult-omni-open-with-prompt " ")

;;; Pick you favorite autosuggest command. (setq consult-omni-default-autosuggest-command #'consult-omni-dynamic-brave-autosuggest) ;;or any other autosuggest source you define

;;; Set your shorthand favorite interactive command (setq consult-omni-default-interactive-command #'consult-omni-multi)

;;; Optionally Set back-end for notes search to ripgrep-all (requires ripgrep-all) ;; (setq consult-omni-notes-backend-command "rga")

;;; Optionally add more interactive commands

;; consult-omni-web (defvar consult-omni-web-sources (list "gptel" "Brave" "elfeed" "mu4e" "Wikipedia" "GitHub" "Invidious" )) (defun consult-omni-web (&optional initial prompt sources no-callback &rest args) "Interactive web search”

This is similar to consult-omni-multi', but runs the search on web sources defined inconsult-omni-web-sources'. See `consult-omni-multi' for more details. " (interactive "P") (let ((prompt (or prompt (concat "[" (propertize "consult-omni-web" 'face 'consult-omni-prompt-face) "]" " Search: "))) (sources (or sources consult-omni-web-sources))) (consult-omni-multi initial prompt sources no-callback args)))

;; consult-omni-local (defvar consult-omni-local-sources (list "ripgrep" "mdfind" "Notes Search" "Apps" "Org Agenda")) (defun consult-omni-local (&optional initial prompt sources no-callback &rest args) "Interactive local search”

This is similar to consult-omni-multi', but runs the search on local sources defined inconsult-omni-local-sources'. See `consult-omni-multi' for more details. " (interactive "P") (let ((prompt (or prompt (concat "[" (propertize "consult-omni-local" 'face 'consult-omni-prompt-face) "]" " Search: "))) (sources (or sources consult-omni-local-sources))) (consult-omni-multi initial prompt sources no-callback args)))

;; consult-omni-scholar (setq consult-omni-scholar-sources (list "PubMed" "Scopus" "Notes Search" "gptel"))

(defun consult-omni-scholar (&optional initial prompt sources no-callback &rest args) "Interactive “multi-source acadmic literature” search

This is similar to consult-omni-multi', but runs the search on academic literature sources defined inconsult-omni-scholar-sources'. See `consult-omni-multi' for more details. " (interactive "P") (let ((prompt (or prompt (concat "[" (propertize "consult-omni-multi" 'face 'consult-omni-prompt-face) "]" " Search: "))) (sources (or sources consult-omni-scholar-sources))) (consult-omni-multi initial prompt sources no-callback args)))

;; AutoSuggest at point (defun consult-omni-autosuggest-at-point () (interactive) (let ((input (or (thing-at-point 'url) (thing-at-point 'filename) (thing-at-point 'symbol) (thing-at-point 'sexp) (thing-at-point 'word)))) (when (and (minibuffer-window-active-p (selected-window)) (equal (substring input 0 1) (consult--async-split-initial nil))) (setq input (substring input 1))) (consult-omni-brave-autosuggest input))))

+end_src

• Comparison to Other Packages To the best of my knowledge no other Emacs package provides the functionality and versatility of consult-omni. While there are several packages for web searches, (see the discussion under consult-web package), there is no omni-search package that I know of.

Outside Emacs, there are of course tolls like MacOS Spotlight and Alfred, that provide omni-search utilities, but compared to =consult-omni= they lack versatility, and customizability.

• Spotlight: spotlight lacks advanced features and is not very customizable. For example it is not easy to access browser history or add an AI chatbot to search results. It's not even possible to get web search results directly in the spotlight. Instead, it suggests opening a web browser and jumping to the search homepage.

• Alfred: Alfred does provide more customizability and advanced workflows, but that does not come for free. all advanced workflows need a paid license. The consistency of the overall workflow is not great either since plugins are made by various people and do not require to follow a consistent pattern (in terms of keybindings, ...).

• Bug Reports To report bug, first check if it is already reported in the [[https://github.com/armindarvish/consult-omni/issues][*issue tracker*]] and see if there is an existing solution or add relevant comments and discussion under the same issue. If not file a new issue following these steps:

1. Make sure the dependencies are installed, and both =consult= and =url-retrieve= (or other relevant commands) work as expected.

2. Remove the package and install the latest version (along with dependencies) and see if the issue persists.

3. In a bare bone vanilla Emacs (>=28) (e.g. =emacs -Q=), install the latest version of consult-omni (and its dependencies) without any configuration or other packages and see if the issue still persists.

4. File an issue and provide important information and context in as much detail as possible in your bug report. Important information can include:

   • Your operating system, version of Emacs (or the version of emacsen you are using), version of consult (see [[https://github.com/emacsorphanage/pkg-info][pkg-info]]).
   • The installation method and the configuration you are using with your consult-omni.
   • If there is an error message, turn debug-on-error on (by =M-x toggle-debug-on-error=) and include the backtrace content in your report.
   • If you are using consult-omni's built-in url-retrieve (e.g. =consult-omni-url-retrieve-sync=) , you can change =consult-omni-log-level= to ='debug=, and inspect the log buffer (hidden buffer called " consult-omni-log" or other name set in =consult-omni-log-buffer-name=). If you chose to include this information in your issue, please make sure personal information and secrets (like API keys) are not exposed.
   • If the error only exists when you have some other packages installed, list those packages (e.g. problem happens when evil is installed)

• Contributions This is an open source package, and I appreciate feedback, suggestions, ideas, etc. There are lots of functionalities or sources that can be added to this package to improve different user's workflows, so if you have some ideas, feel free to file an issue for a feature request.

I would love to see help from other contributors both for improving/maintaining current sources/features as well as for working on new sources or features. If you are interested and want to contribute to the code, please note that the main branch is currently stable (as stable as a work in progress like this can be) and the develop branch is the current work in progress. So, start from the develop branch to get the latest work-in-progress updates and create a new branch with names such as feature/name-of-the-feature or fix/issue, ... Do the edits and then create a new pull request to merge back with the develop branch when you are done with your edits.

Importantly, keep in mind that I am using a literate programming approach where everything goes into a single source of truth consult-omni.org and then gets tangled to appropriate files (e.g. consult-omni.el). If you open a pull-request where you directly edited the .el files, I will likely not approve it because that will then get overwritten later when I tangle from the .org file. In other words, Do Not Edit The .el Files! only edit the .org file and tangle to .el files. Keep in mind that while literate programming may not make a lot of sense in other scenarios, in this case, it really helps with adding new sources/features by copying from other examples (since everything is in one org file), therefore in this case I insist o using the literate programming.

• Acknowledgments

Obviously this package would not have been possible without the fabulous [[https://github.com/minad/consult][consult]] and [[https://github.com/oantolin/embark][embark]] packages. It also took inspiration from other packages including but not limited to the predecessor [[https://github.com/mnewt/counsel-web][counsel-web]], as well as [[https://github.com/hrs/engine-mode][engine-mode]], [[https://github.com/Malabarba/emacs-google-this][emacs-google-this]], [[https://github.com/emacs-helm/helm][helm]].

Furthermore I would like to thank [[https://github.com/karthink][@karthink]], and [[https://github.com/armindarvish/consult-omni][@minad]] for their fabulous suggestions, discussions, contributions, etc. This package would not have been possible without [[https://github.com/karthink][@karthink]]'s help in prototyping and ideation (see [[https://github.com/karthink/consult-web-mini][consult-web-mini]]) and [[https://github.com/armindarvish/consult-omni][@minad]]'s precious comments and suggestions.