+author: Shariff AM Faleel

+language: en

+HTML:

org-roam-ql
org-roam-ql-ql

+TITLE: org-roam-ql - query language for org-roam

This package provides an interface to easily query and display results from your ~org-roam~ database.

• Contents

• [[#screen-shots][Screen-shots]]

• [[#installationSetup][Installation/Setup]]

• [[#usage][Usage]]

  • [[#installationsetup][Installation/Setup]]
  • [[#valid-values-for-source-or-query][Valid values for SOURCE-OR-QUERY]]
  • [[#adding-new-predicates][Adding new predicates]]
  • [[#org-dynamic-block][Org dynamic block]]

• [[#working-with-org-ql][Working with org-ql]]

• Screen-shots

You can query org-roam with ~org-roam-ql-search~. The results are displayed in an org-roam-like buffer. org-roam-ql also comes with a transient that can be used to modify the results viewed. The transient can be activated with ~v~. You can modify the title (~t~), query (~q~), sort (~s~) and specify if the query is a subquery (apply query on the results of the buffer) or query against the whole org-roam database (~i~). Refreshing the buffer (~r~) will display the updated results.

[[images/demo4.gif]]

The transient is available in the org-roam buffer as well, this allows you to start a query from the results in the org-roam buffer. You also can view the results in an agenda-like buffer (~S~). When entering a query in any interactive function of org-roam-ql, it also has completion-at-point. ~org-roam-ql~ buffer also supports bookmark, i.e., you can set a bookmark by calling ~bookmark-set~. For convenience, the transient also has a shortcut for that (~b~).

[[images/demo6.gif]]

• Installation/Setup ~org-roam-ql~ can be installed from [[https://melpa.org/#/org-roam-ql][MELPA]] or with other package management tools like [[https://framagit.org/steckerhalter/quelpa][quelpa]] and [[https://github.com/radian-software/straight.el][straight]]

Example configuration:

+begin_src emacs-lisp

(use-package org-roam-ql ;; If using straight :straight (org-roam-ql :type git :host github :repo "ahmed-shariff/org-roam-ql" :files (:defaults (:exclude "org-roam-ql-ql.el"))) ;; If using quelpa :quelpa (org-roam-ql :fetcher github :repo "ahmed-shariff/org-roam-ql" :files (:defaults (:exclude "org-roam-ql-ql.el"))) ;; Simple configuration :after (org-roam) :bind ((:map org-roam-mode-map ;; Have org-roam-ql's transient available in org-roam-mode buffers ("v" . org-roam-ql-buffer-dispatch) :map minibuffer-mode-map ;; Be able to add titles in queries while in minibuffer. ;; This is similar to `org-roam-node-insert', but adds ;; only title as a string. ("C-c n i" . org-roam-ql-insert-node-title))))

+end_src

• Usage ** Commands/functions

• =org-roam-ql-search (SOURCE-OR-QUERY &optional TITLE SORT-FN)= :: This is an interactive command that creates a ~org-roam-ql~ buffer with the nodes of the corresponding [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] with ~TITLE~. An ~org-roam-ql~ buffer is functionally similar to the ~org-roam-buffer~, but allows displaying any list of nodes ([[#screen-shots][see screen-shots above]]). When called interactively, it will prompt for the ~SOURCE-OR-QUERY~ and ~TITLE~. Note that when entering queries interactively either in ~org-roam-ql-search~ or in the transient, you can get completion-at-point with ~tab~. ~SORT-FN~ is used for sorting the results. It can be a string name of a registered sort function or a predicate function that can be used to sort the nodes (should take two nodes as input and return a non-nil value if the first node should be before the second). By default the following sort function are registered: ~file-mtime~, ~file-atime~, ~deadline~, ~scheduled~, ~point~, ~level~, ~file-title~, ~file~ and ~title~. Each corresponds to the respective slot of an org-roam-node. It is possible to register new sort functions with ~org-roam-ql-register-sort-fn~. These registered functions will also appear as options for completion in the transient.

• =org-roam-ql-nodes (SOURCE-OR-QUERY)= :: Given a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] , return a list of nodes.

• ~org-roam-ql-agenda-block (QUERY)~ :: Meant to be used in ~org-agenda-custom-commands~ as a user-defined function. Insert items from processing ~QUERY~ (which is a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]]) into current buffer. QUERY is the `match' item in the custom command form. Currently this doesn't respect agenda restrict. Example:

  +begin_src emacs-lisp

  (setq org-agenda-custom-commands ("cr" "Node a" org-roam-ql-agenda-block '(title "Node a")))

  +end_src

• ~org-roam-ql-nodes-files (SOURCE-OR-QUERY)~ :: Given a [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]] , returns a list of files of the nodes. Can be used in ~org-agenda-custom-commands~. Example:

  +begin_src emacs-lisp

  (setq org-agenda-custom-commands ("cr" "todo nodes" todo "TODO" ((org-agenda-files (org-roam-ql-nodes-files '(title "Node"))))))

  +end_src

• ~org-roam-ql-add-saved-query (NAME DOCSTRING QUERY)~ :: Stores valid [[#valid-values-for-source-or-query][QUERY]] with ~NAME~ and ~DOCSTRING~. The ~NAME~ can be used as a query in place of any other ~SOURCE-OR-QUERY~. ~NAME~ can be a string or a symbol. ** Valid values for ~SOURCE-OR-QUERY~

• A list of ~org-roam-nodes~ :: This should self explanatory.

• A list of parameters that can be passed to ~org-roam-db-query~ :: It should be a list of the form ~(QUERY ARG1 ARG2...)~. The result of calling ~org-roam-db-query~ with these parameters should return a list of records where the first element is the ID of a corresponding node. For example:

  +begin_src emacs-lisp

  (org-roam-ql-nodes '([:select [id] :from nodes :where (= todo \"TODO\")]))

  +end_src

• Saved query name :: Name of a saved query (see ~org-roam-ql-add-saved-query~). This can be string or a symbol. Note that the name of the saved query is always stored as a symbol. If a string is passed, it gets interned into a symbol before looking up the query. The returned nodes will be a result of executing the query represented by the name.

• Bookmark name :: Name of a bookmark of a org-roam-ql-buffer. This matched against the name given when ~bookmark-set~ is called from a org-roam-ql-buffer.

• Buffer name :: A buffer or buffer-name of a ~org-roam~ buffer, a ~org-roam-ql~ buffer or an agenda-like buffer displaying a list of org-roam nodes.

• Function :: A function that returns a list of ~org-roam-nodes~

• A ~QUERY~ :: This is a predicate, similar to the predicates in [[https://github.com/alphapapa/org-ql][org-ql]]. Returns all nodes that pass for the given predicate. For example, consider the following call to ~org-roam-ql-nodes~:

  +begin_src emacs-lisp

  (org-roam-ql-nodes '(and (todo "TODO") (tags "tag1" "tag2") "org-roam"))

  +end_src

  In the above example, the result would contain any nodes whose todo state is =TODO=, have tags "tag1" and "tag2" and are in the org-roam buffer. The following are predicates available by default in org-roam-ql:

  • =or (SOURCE-OR-QUERY1 SOURCE-OR-QUERY2 ...)= :: Tests if a node matches/contained-in any of the ~SOURCE-OR-QUERY~'s.
  • =and (SOURCE-OR-QUERY1 SOURCE-OR-QUERY2 ...)= :: Similar to ~or~, but should satisfy all predicates or contained in all the results of ~SOURCE-OR-QUERY~'s.
  • =not (SOURCE-OR-QUERY)= :: Tests if a node doesn't match the result or not contained in the result of ~SOURCE-OR-QUERY~.
  • =file (REGEXP &optional EXACT)= :: Test if nodes file name matches ~REGEXP~. If ~EXACT~ is non-nil, the file slot should be an exact match to ~REGEXP~. Note the slot ~file~ of an org-roam-node would contain the absolute path.
  • =file-title (REGEXP &optional EXACT)= :: Similar to ~file~, tests the ~file-title~ slot of a node.
  • =id (ID)= :: Tests if the ~ID~ of a node is a match to the value passed.
  • =level (LEVEL)= :: Tests if the ~level~ of a node is equal to ~LEVEL~.
  • =todo (REGEXP &optional EXACT)= :: Similar to ~file~, tests the todo state of a node.
  • =priority (REGEXP &optional EXACT)= :: Similar to ~file~, tests the priority of a node.
  • =scheduled (COMPARISON TIME-STRING)= :: Compares the ~scheduled~ of the node with ~TIME-STAMP~ based on ~COMPARISON~. ~TIME-STAMP~ is any valid value for [[https://orgmode.org/manual/The-date_002ftime-prompt.html][org date-time prompt]]. ~COMPARISON~ is either ~<~ or ~>~. Example: ~(scheduled > "-3w")~
  • =deadline= (COMPARISON TIME-STRING) :: Same as ~scheduled~, tests the ~deadline~ of a node.
  • =title (REGRXP &optional EXACT)= :: Similar to ~file~, tests the title of a node.
  • =properties (PROP PROP-VAL)= :: Tests if the value of the property of a node PROP is a match to PROP-VAL. PROP-VAL can be a regular expression.
  • =tags (TAG1 TAG2 ...)= :: Tests if the tags of a node have TAG1, TAG2, etc.
  • =refs (REGEXP &optional EXACT)= :: Similar to ~file~, tests the nodes refs slot.
  • =backlink-to (SOURCE-OR-QUERY)= :: Tests if the node has a backlink to any of the nodes from the results ~SOURCE-OR-QUERY~.
  • =backlink-from (SOURCE-OR-QUERY)= :: Similar to ~backlink-to~, tests if there are any backlinks from (aka forwardlinks) the resulting nodes from ~SOURCE-OR-QUERY~.
  • =in-buffer (BUFFER-NAME)= :: This is similar to passing a buffer-name as ~SOURCE-OR-QUERY~. Tests if a node is in the org-roam buffer named ~BUFFER-NAME~.
  • =nodes-list (NODES-LIST)= :: This is similar to passing a list of nodes as ~SOURCE-OR-QUERY~. Tests if a node is in the ~NODES-LIST~.
  • =function (FUNC)= :: This is similar to passing a function as ~SOURCE-OR-QUERY~. Tests if the node is in the result of executing the function ~FUNC~.
  • =funcall (FUNC)= :: Tests a node with the function ~FUNC~, which takes an org-roam node as parameter. Test passes if the function returns non-nil. ** Adding new predicates There are two ways to add a new predicate to org-roam-ql:

• =org-roam-ql-defpred (NAME DOCSTRING EXTRACTION-FUNCTION COMPARISON-FUNCTION)= :: Creates a predicate that can be used as ~SOURCE-OR-QUERY~. For example, for a predicate defined as follows:

  +begin_src emacs-lisp

  (org-roam-ql-defpred sample "A sample predicate" extraction-function comparison-function)

  +end_src

  When the following predicate is used as ~SOURCE-OR-QUERY~ :

  +begin_src emacs-lisp

  (org-roam-ql-nodes '(sample arg1 arg2))

  +end_src

  It tests each ~node~ in the whole org-roam database as follows:

  +begin_src emacs-lisp

  (apply comparison-function (append (list (funcall extraction-function node)) arg1 arg2))

  +end_src

  The ~EXTRACTION-FUNCTION~ takes an org-roam-node and returns a value that will be passed as the first parameter to ~COMPARISON-FUNCTION~. The remainder of the parameters when calling the predicate is passed as remaining parameters to ~COMPARISON-FUNCTION~. When the ~COMPARISON-FUNCTION~ returns a non-nil value, it will be included in the result.

• =org-roam-ql-defexpansion (NAME DOCSTRING EXPANSION-FUNCTION)= :: Adds an ~EXPANSION-FUNCTION~ which will be identified by ~NAME~ in a org-roam-ql query. The ~EXPANSION-FUNCTION~ should take the parameters passed in the query and return values that can be passed to ~org-roam-nodes~.

** Adding a sorting function

• =org-roam-ql-register-sort-fn (FUNCTION-NAME SORT-FUNCTION)= :: Registers a sort function which can be used with ~org-roam-ql-nodes~. ~FUNCTION-NAME~ is the string name used to refer to this function with. ~SORT-FUNCTION~ is a function that takes two org-roam-nodes and return a truth value, which is used to sort, i.e., if non-nil, the first node would be before the second node passed to the function. Uses `seq-sort'. If a sort-function with the given name already exists, it would be overwritten. The following example registers a sort function named "custom-prop" which sorts the values based on the "CUSTOM-PROP" property of a node.

  +begin_src emacs-lisp

  (org-roam-ql-register-sort-fn "custom-prop" (lambda (el1 el2) (string< (cdr (assoc "CUSTOM-PROP" (org-roam-node-properties el1))) (cdr (assoc "CUSTOM-PROP" (org-roam-node-properties el2))))))

  +end_src

** Org dynamic block Similar to ~org-ql~, ~org-roam-ql~ also provides a dynamic block. The header parameters are as follows:

• ~:query~ - A valid [[#valid-values-for-source-or-query][ ~SOURCE-OR-QUERY~ ]]
• ~:columns~ - A list of columns to display. Each column name is a slot name of ~org-roam-nodes~. For any function/accessor with a name of the form ~org-roam-node-~, which takes an org-roam-node as a parameter, ~~ can also be used column name. For example, if there is a function named ~org-roam-node-short-title~, ~short-title~ can be used as a column name, this will result in a column with the title ~short-title~ where the content of each row is the result of calling the respective function.
• ~:sort~ - Name of a registered sort functions. See [[#commandsfunctions][ ~org-roam-ql-search~ ]] for more info on the values for sort functions.
• ~:take~ (optional) - If a positive integer N, take the first N elements, if a negative -N, take the last N nodes.
• ~:no-link~ (optional) - If a non-nil value is set, the first column containing the links will be dropped.

If no-link is not provided as a parameter, the first column is a link to the node. Since it is an id link, it will be a backlink to the node.

Following is an example of a dynamic block and its result.

[[file:images/dynamic-block.jpg]]

• Working with org-ql Optionally, ~org-roam-ql~ results can be visualized with [[https://github.com/alphapapa/org-ql][org-ql]], available through the extension ~org-roam-ql-ql~ (naming things is hard!!). This also can be installed from [[https://melpa.org/#/org-roam-ql-ql][MELPA]] or with other package management tools like [[https://framagit.org/steckerhalter/quelpa][quelpa]] and [[https://github.com/radian-software/straight.el][straight]].

+begin_src emacs-lisp

(use-package org-roam-ql-ql ;; If using straight :straight (org-roam-ql-ql :type git :host github :repo "ahmed-shariff/org-roam-ql" :files (:defaults (:exclude "org-roam-ql.el"))) ;; If using quelpa :quelpa (org-roam-ql-ql :fetcher github :repo "ahmed-shariff/org-roam-ql" :files (:defaults (:exclude "org-roam-ql.el"))) ;; Simple config :after (org-ql org-roam-ql) :config (org-roam-ql-ql-init))

+end_src

Note that org-ql works only with org entries, i.e., heading nodes. Hence, if there are any file nodes in the result, they will not be displayed. To be clear about that, when org-roam-ql results are displayed in an org-ql-view buffer, a warning is added to the end mentioning how many file nodes were there in the result. If the extension is loaded, you may view the org-roam-ql results with ~Q~ from the org-roam-ql transient. An org-ql-view can be viewed in an org-roam-like buffer with ~R~ from the org-ql-view transient.

[[images/demo5.gif]]