Testing new taxy-based org-ql-report view (feedback welcome)

[alphapapa]

This issue is for tracking early feedback for testers of the https://github.com/alphapapa/org-ql/tree/wip/taxy-org-ql-view branch.

To test:

1. Install the package from the branch, e.g. with Quelpa:

   (use-package org-ql
   :quelpa (org-ql :fetcher github :repo "alphapapa/org-ql" :branch "wip/taxy-org-ql-view"
           :files (:defaults (:exclude "helm-org-ql.el"))))

   It may be helpful to use https://github.com/alphapapa/with-emacs.sh to install it into a separate Emacs configuration, but this is optional.

2. Load the library, e.g. by evaluating (require 'taxy-org-ql-view).

3. Evaluate a taxy-org-ql-report form, for example:

   
   ;; This additional predicate is used in the example below:
   (org-ql-defpred stuck ()
   "Stuck projects."
   :normalizers ((`(,predicate-names . ,args)
                `(and (todo "PROJECT") (not (descendants (todo "NEXT" "WAITING")))))))

(taxy-org-ql-report :buffer "Clients" :queries '(( :name "Agenda" :where (and (or (scheduled) (deadline auto)) (not (done))) :group (((planned :today :name "Today")) ((planned :past)) ((planned :future)))) ( :name "Stuck" :where (stuck)) ( :name "UNDERWAY" :where (todo "UNDERWAY" "NOW")) ( :name "NEXT" :where (todo "NEXT")) ( :name "Neglected" :where (and (todo "PROJECT") (not (descendants (and (todo "WAITING") (scheduled)))) (not (or (clocked :from -7) (descendants (clocked :from -7)))) ;; Prevents stuck projects from being duplicated in neglected list. (not (stuck))) :group nil)) :sort '(priority date reverse) :group '(priority) :columns '("Keyword" "Heading\Parent" "Pri" "Planning" "Tags") :sections `(( :name "Client 1" :from "~/client1.org") ( :name "Client 2" :from "~/client2.org")))

The documentation is not yet written, but the bottom line is that this displays the results of multiple org-ql queries in a single buffer, similar to Org Agenda blocks.  In this example, all of the queries are run for each section, each of which runs on a different file.  The result looks something like this, where each section is a collapsible `magit-section`:

![org-ql-report](https://user-images.githubusercontent.com/601365/222636055-52eb2ad2-4f6d-4abe-9790-9b183c569543.png)

One of the primary new features is the use of `taxy` to dynamically group entries.  For example, you could use:
```el
:group '((planned) todo)

...which would put all items with planning timestamps in one group, and group remaining items by their to-do keywords. You could also give an argument to the to-do matcher, like :group '((planned) ((todo "SOMEDAY" "MAYBE")) priority), which would only collect items with the SOMEDAY or MAYBE keywords, and then group remaining items by priority.

Among other things, this branch will also allow easier customization of the output format. For example, the :columns argument is used to select columns for display, each of which is defined with simple top-level forms here: https://github.com/alphapapa/org-ql/blob/9496f7a8fcba11b244ea0c1921a3c6dcec7a02fe/taxy-org-ql-view.el#L106 And users can define additional columns in their own configurations and use them in the report view.

Also, the taxy-org-ql-report view retains some compatibility with Org Agenda commands since it continues to add relevant text properties to each item (e.g. you can use C-c C-d to change the deadline of an item). In the future I may design a new system for defining commands for this kind of buffer, but for now this is still useful.

Any feedback on this branch is welcome. I don't intend to remove the existing org-ql-view library in the next release, but someday it may be subsumed by this one.

[ParetoOptimalDev]

I tried the above and got this error with empty files at ~/client1.org and ~/client2.org. I'm unsure if that's expected behavior at this juncture, but here's the traceback in case it's not from GNU Emacs 30.0.50:

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
  1+(nil)
  (format " %%%s%ss" first-column-align (1+ (cdr (car column-sizes))))
  (format (format " %%%s%ss" first-column-align (1+ (cdr (car column-sizes)))) (car (car column-sizes)))
  (concat (format (format " %%%s%ss" first-column-align (1+ (cdr (car column-sizes)))) (car (car column-sizes))) (let* ((--cl-var-- (cdr column-sizes)) (size nil) (name nil) (column-alist nil) (align nil) (spec nil) (--cl-var-- "") (--cl-var-- t)) (while (consp --cl-var--) (progn (setq size (car --cl-var--)) (setq name (car-safe (prog1 size (setq size ...))))) (setq column-alist (alist-get name formatters nil nil #'equal)) (setq align (let* ((val (alist-get ... column-alist))) (cond ((memq val ...) (let nil "-")) ((eq val ...) (let nil "")) (t (let ... ...))))) (setq spec (format " %%%s%ss" align size)) (setq --cl-var-- (concat --cl-var-- (format spec name))) (setq --cl-var-- (cdr --cl-var--)) (setq --cl-var-- nil)) --cl-var--))
  (let* ((first-column-name (car (car column-sizes))) (first-column-alist (alist-get first-column-name formatters nil nil #'equal)) (first-column-align (let* ((val (alist-get 'align first-column-alist))) (cond ((memq val '...) (let nil "-")) ((eq val 'right) (let nil "")) (t (let (...) (error "No clause matching `%S'" x588))))))) (concat (format (format " %%%s%ss" first-column-align (1+ (cdr (car column-sizes)))) (car (car column-sizes))) (let* ((--cl-var-- (cdr column-sizes)) (size nil) (name nil) (column-alist nil) (align nil) (spec nil) (--cl-var-- "") (--cl-var-- t)) (while (consp --cl-var--) (progn (setq size (car --cl-var--)) (setq name (car-safe (prog1 size ...)))) (setq column-alist (alist-get name formatters nil nil #'equal)) (setq align (let* ((val ...)) (cond (... ...) (... ...) (t ...)))) (setq spec (format " %%%s%ss" align size)) (setq --cl-var-- (concat --cl-var-- (format spec name))) (setq --cl-var-- (cdr --cl-var--)) (setq --cl-var-- nil)) --cl-var--)))
  taxy-magit-section-format-header(nil (("Tags" (align) (formatter . org-ql-view-column-format-tags)) ("Planning" (align) (formatter . org-ql-view-column-format-planning)) ("Pri" (align) (formatter . org-ql-view-column-format-pri)) ("Heading\\Parent" (align) (formatter . org-ql-view-column-format-heading\\parent)) ("Heading" (align) (formatter . org-ql-view-column-format-heading)) ("Keyword" (align . right) (formatter . org-ql-view-column-format-keyword)) ("Category" (align . right) (formatter . org-ql-view-column-format-category))))
  (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters))
  (progn (setq format-cons (taxy-org-ql-view-magit-section-format-items org-ql-view-columns org-ql-view-column-formatters taxy-org-ql-view-taxy :table taxy-org-ql-view-format-table)) (setq column-sizes (cdr format-cons)) (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters)))
  (let ((inhibit-read-only t) (taxy-magit-section-insert-indent-items nil)) (erase-buffer) (progn (setq format-cons (taxy-org-ql-view-magit-section-format-items org-ql-view-columns org-ql-view-column-formatters taxy-org-ql-view-taxy :table taxy-org-ql-view-format-table)) (setq column-sizes (cdr format-cons)) (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters))) (add-face-text-property 0 (length header-line-format) 'org-ql-view-header-line nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items 'first :initial-depth -1) (goto-char (point-min)))
  (save-current-buffer (set-buffer buffer) (if append nil (taxy-org-ql-view-mode) (setq taxy-org-ql-view-taxy (make-taxy-magit-section :name (propertize (buffer-name buffer) 'face 'taxy-org-ql-view-header) :format-fn --cl-format-item--))) (setq taxy-org-ql-view-args (if (member rest taxy-org-ql-view-args) taxy-org-ql-view-args (cons rest taxy-org-ql-view-args))) (if columns (progn (set (make-local-variable 'org-ql-view-columns) columns))) (let ((tail queries)) (while tail (let ((x598 (car tail))) (progn (ignore (mapp x598)) (let* ((x599 ...) (x600 ...) (x601 ...) (x602 ...) (x603 ...) (x604 ...)) (let (... ... ... ... ... ...) (progn ... ... ...)))) (setq tail (cdr tail))))) (progn (progn (or (progn (and (memq (type-of instance-taxy) cl-struct-taxy-tags) t)) (signal 'wrong-type-argument (list 'taxy instance-taxy))) (let* ((v instance-taxy)) (aset v 5 (nreverse (progn (or ... ...) (aref instance-taxy 5)))))) (progn (or (progn (and (memq (type-of taxy-org-ql-view-taxy) cl-struct-taxy-tags) t)) (signal 'wrong-type-argument (list 'taxy taxy-org-ql-view-taxy))) (let* ((v taxy-org-ql-view-taxy)) (aset v 5 (append (progn (or ... ...) (aref taxy-org-ql-view-taxy 5)) (list instance-taxy)))))) (let ((inhibit-read-only t) (taxy-magit-section-insert-indent-items nil)) (erase-buffer) (progn (setq format-cons (taxy-org-ql-view-magit-section-format-items org-ql-view-columns org-ql-view-column-formatters taxy-org-ql-view-taxy :table taxy-org-ql-view-format-table)) (setq column-sizes (cdr format-cons)) (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters))) (add-face-text-property 0 (length header-line-format) 'org-ql-view-header-line nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items 'first :initial-depth -1) (goto-char (point-min))) (pop-to-buffer (current-buffer)))
  (let (--cl-make-fn--) (setq --cl-make-fn-- #'(lambda (&rest args) "\n\n(fn &rest ARGS)" (apply #'make-taxy-magit-section :make --cl-make-fn-- :take (taxy-make-take-function make-fn-group taxy-org-ql-view-keys) :format-fn --cl-format-item-- :heading-face-fn --cl-heading-face-- :level-indent org-ql-view-level-indent :item-indent org-ql-view-item-indent args))) (save-current-buffer (set-buffer buffer) (if append nil (taxy-org-ql-view-mode) (setq taxy-org-ql-view-taxy (make-taxy-magit-section :name (propertize (buffer-name buffer) 'face 'taxy-org-ql-view-header) :format-fn --cl-format-item--))) (setq taxy-org-ql-view-args (if (member rest taxy-org-ql-view-args) taxy-org-ql-view-args (cons rest taxy-org-ql-view-args))) (if columns (progn (set (make-local-variable 'org-ql-view-columns) columns))) (let ((tail queries)) (while tail (let ((x598 (car tail))) (progn (ignore (mapp x598)) (let* (... ... ... ... ... ...) (let ... ...))) (setq tail (cdr tail))))) (progn (progn (or (progn (and (memq ... cl-struct-taxy-tags) t)) (signal 'wrong-type-argument (list 'taxy instance-taxy))) (let* ((v instance-taxy)) (aset v 5 (nreverse (progn ... ...))))) (progn (or (progn (and (memq ... cl-struct-taxy-tags) t)) (signal 'wrong-type-argument (list 'taxy taxy-org-ql-view-taxy))) (let* ((v taxy-org-ql-view-taxy)) (aset v 5 (append (progn ... ...) (list instance-taxy)))))) (let ((inhibit-read-only t) (taxy-magit-section-insert-indent-items nil)) (erase-buffer) (progn (setq format-cons (taxy-org-ql-view-magit-section-format-items org-ql-view-columns org-ql-view-column-formatters taxy-org-ql-view-taxy :table taxy-org-ql-view-format-table)) (setq column-sizes (cdr format-cons)) (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters))) (add-face-text-property 0 (length header-line-format) 'org-ql-view-header-line nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items 'first :initial-depth -1) (goto-char (point-min))) (pop-to-buffer (current-buffer))))
  (let* ((--cl-add-props-- #'(lambda (plist) (let ((tail ...)) (while tail (let ... ... ...)) plist))) (--cl-format-item-- #'(lambda (item) (let* ((string ...) (marker ...)) (propertize string 'org-hd-marker marker 'org-marker marker)))) (--cl-heading-face-- #'(lambda (depth) (cond ((eql depth -1) (let nil ...)) ((eql depth 0) (let nil ...)) ((eql depth 1) (let nil ...)) (t (let nil ...)))))) (let (--cl-make-fn--) (setq --cl-make-fn-- #'(lambda (&rest args) "\n\n(fn &rest ARGS)" (apply #'make-taxy-magit-section :make --cl-make-fn-- :take (taxy-make-take-function make-fn-group taxy-org-ql-view-keys) :format-fn --cl-format-item-- :heading-face-fn --cl-heading-face-- :level-indent org-ql-view-level-indent :item-indent org-ql-view-item-indent args))) (save-current-buffer (set-buffer buffer) (if append nil (taxy-org-ql-view-mode) (setq taxy-org-ql-view-taxy (make-taxy-magit-section :name (propertize (buffer-name buffer) 'face 'taxy-org-ql-view-header) :format-fn --cl-format-item--))) (setq taxy-org-ql-view-args (if (member rest taxy-org-ql-view-args) taxy-org-ql-view-args (cons rest taxy-org-ql-view-args))) (if columns (progn (set (make-local-variable 'org-ql-view-columns) columns))) (let ((tail queries)) (while tail (let ((x598 ...)) (progn (ignore ...) (let* ... ...)) (setq tail (cdr tail))))) (progn (progn (or (progn (and ... t)) (signal 'wrong-type-argument (list ... instance-taxy))) (let* ((v instance-taxy)) (aset v 5 (nreverse ...)))) (progn (or (progn (and ... t)) (signal 'wrong-type-argument (list ... taxy-org-ql-view-taxy))) (let* ((v taxy-org-ql-view-taxy)) (aset v 5 (append ... ...))))) (let ((inhibit-read-only t) (taxy-magit-section-insert-indent-items nil)) (erase-buffer) (progn (setq format-cons (taxy-org-ql-view-magit-section-format-items org-ql-view-columns org-ql-view-column-formatters taxy-org-ql-view-taxy :table taxy-org-ql-view-format-table)) (setq column-sizes (cdr format-cons)) (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters))) (add-face-text-property 0 (length header-line-format) 'org-ql-view-header-line nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items 'first :initial-depth -1) (goto-char (point-min))) (pop-to-buffer (current-buffer)))))
  (let ((buffer (cond ((bufferp buffer) buffer) ((stringp buffer) (or (get-buffer buffer) (get-buffer-create (format "*Taxy Org QL View: %s*" buffer)))))) (instance-taxy (make-taxy-magit-section :name name)) format-cons column-sizes make-fn-group) (let* ((--cl-add-props-- #'(lambda (plist) (let (...) (while tail ...) plist))) (--cl-format-item-- #'(lambda (item) (let* (... ...) (propertize string ... marker ... marker)))) (--cl-heading-face-- #'(lambda (depth) (cond (... ...) (... ...) (... ...) (t ...))))) (let (--cl-make-fn--) (setq --cl-make-fn-- #'(lambda (&rest args) "\n\n(fn &rest ARGS)" (apply #'make-taxy-magit-section :make --cl-make-fn-- :take (taxy-make-take-function make-fn-group taxy-org-ql-view-keys) :format-fn --cl-format-item-- :heading-face-fn --cl-heading-face-- :level-indent org-ql-view-level-indent :item-indent org-ql-view-item-indent args))) (save-current-buffer (set-buffer buffer) (if append nil (taxy-org-ql-view-mode) (setq taxy-org-ql-view-taxy (make-taxy-magit-section :name (propertize ... ... ...) :format-fn --cl-format-item--))) (setq taxy-org-ql-view-args (if (member rest taxy-org-ql-view-args) taxy-org-ql-view-args (cons rest taxy-org-ql-view-args))) (if columns (progn (set (make-local-variable ...) columns))) (let ((tail queries)) (while tail (let (...) (progn ... ...) (setq tail ...)))) (progn (progn (or (progn ...) (signal ... ...)) (let* (...) (aset v 5 ...))) (progn (or (progn ...) (signal ... ...)) (let* (...) (aset v 5 ...)))) (let ((inhibit-read-only t) (taxy-magit-section-insert-indent-items nil)) (erase-buffer) (progn (setq format-cons (taxy-org-ql-view-magit-section-format-items org-ql-view-columns org-ql-view-column-formatters taxy-org-ql-view-taxy :table taxy-org-ql-view-format-table)) (setq column-sizes (cdr format-cons)) (setq header-line-format (taxy-magit-section-format-header column-sizes org-ql-view-column-formatters))) (add-face-text-property 0 (length header-line-format) 'org-ql-view-header-line nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items 'first :initial-depth -1) (goto-char (point-min))) (pop-to-buffer (current-buffer))))))
  (progn (ignore from group sort append) (let ((buffer (cond ((bufferp buffer) buffer) ((stringp buffer) (or (get-buffer buffer) (get-buffer-create ...))))) (instance-taxy (make-taxy-magit-section :name name)) format-cons column-sizes make-fn-group) (let* ((--cl-add-props-- #'(lambda (plist) (let ... ... plist))) (--cl-format-item-- #'(lambda (item) (let* ... ...))) (--cl-heading-face-- #'(lambda (depth) (cond ... ... ... ...)))) (let (--cl-make-fn--) (setq --cl-make-fn-- #'(lambda (&rest args) "\n\n(fn &rest ARGS)" (apply ... :make --cl-make-fn-- :take ... :format-fn --cl-format-item-- :heading-face-fn --cl-heading-face-- :level-indent org-ql-view-level-indent :item-indent org-ql-view-item-indent args))) (save-current-buffer (set-buffer buffer) (if append nil (taxy-org-ql-view-mode) (setq taxy-org-ql-view-taxy (make-taxy-magit-section :name ... :format-fn --cl-format-item--))) (setq taxy-org-ql-view-args (if (member rest taxy-org-ql-view-args) taxy-org-ql-view-args (cons rest taxy-org-ql-view-args))) (if columns (progn (set ... columns))) (let ((tail queries)) (while tail (let ... ... ...))) (progn (progn (or ... ...) (let* ... ...)) (progn (or ... ...) (let* ... ...))) (let ((inhibit-read-only t) (taxy-magit-section-insert-indent-items nil)) (erase-buffer) (progn (setq format-cons ...) (setq column-sizes ...) (setq header-line-format ...)) (add-face-text-property 0 (length header-line-format) 'org-ql-view-header-line nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items 'first :initial-depth -1) (goto-char (point-min))) (pop-to-buffer (current-buffer)))))))
  (progn (let ((--cl-keys-- rest)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '(:name :buffer :queries :from :where :group :sort :append :columns :narrow :allow-other-keys)) (if (cdr --cl-keys--) nil (error "Missing argument for %s" (car --cl-keys--))) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ... rest))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:name :buffer :que..." (car --cl-keys--)))))) (progn (ignore from group sort append) (let ((buffer (cond ((bufferp buffer) buffer) ((stringp buffer) (or ... ...)))) (instance-taxy (make-taxy-magit-section :name name)) format-cons column-sizes make-fn-group) (let* ((--cl-add-props-- #'(lambda ... ...)) (--cl-format-item-- #'(lambda ... ...)) (--cl-heading-face-- #'(lambda ... ...))) (let (--cl-make-fn--) (setq --cl-make-fn-- #'(lambda ... "\n\n(fn &rest ARGS)" ...)) (save-current-buffer (set-buffer buffer) (if append nil (taxy-org-ql-view-mode) (setq taxy-org-ql-view-taxy ...)) (setq taxy-org-ql-view-args (if ... taxy-org-ql-view-args ...)) (if columns (progn ...)) (let (...) (while tail ...)) (progn (progn ... ...) (progn ... ...)) (let (... ...) (erase-buffer) (progn ... ... ...) (add-face-text-property 0 ... ... nil header-line-format) (taxy-magit-section-insert taxy-org-ql-view-taxy :items ... :initial-depth -1) (goto-char ...)) (pop-to-buffer (current-buffer))))))))
  (let* ((name (car (cdr (plist-member rest ':name)))) (buffer (car (cdr (plist-member rest ':buffer)))) (queries (car (cdr (plist-member rest ':queries)))) (from (car (cdr (plist-member rest ':from)))) (where (car (cdr (plist-member rest ':where)))) (group (car (cdr (plist-member rest ':group)))) (sort (car (cdr (plist-member rest ':sort)))) (append (car (cdr (plist-member rest ':append)))) (columns (car (cdr (plist-member rest ':columns)))) (narrow (car (cdr (plist-member rest ':narrow))))) (progn (let ((--cl-keys-- rest)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '...) (if (cdr --cl-keys--) nil (error "Missing argument for %s" ...)) (setq --cl-keys-- (cdr ...))) ((car (cdr ...)) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:name :buffer :que..." (car --cl-keys--)))))) (progn (ignore from group sort append) (let ((buffer (cond (... buffer) (... ...))) (instance-taxy (make-taxy-magit-section :name name)) format-cons column-sizes make-fn-group) (let* ((--cl-add-props-- #'...) (--cl-format-item-- #'...) (--cl-heading-face-- #'...)) (let (--cl-make-fn--) (setq --cl-make-fn-- #'...) (save-current-buffer (set-buffer buffer) (if append nil ... ...) (setq taxy-org-ql-view-args ...) (if columns ...) (let ... ...) (progn ... ...) (let ... ... ... ... ... ...) (pop-to-buffer ...))))))))
  taxy-org-ql-view(:buffer "Clients" :columns ("Keyword" "Heading\\Parent" "Pri" "Planning" "Tags") :name #("Client 1" 0 8 (face org-ql-view-heading-1)) :from "~/client1.org" :where nil :sort (priority date reverse) :group (priority) :queries ((:name #("Agenda" 0 6 (face org-ql-view-heading-2)) :where (and (or (scheduled) (deadline auto)) (not (done))) :group (((planned :today :name "Today")) ((planned :past)) ((planned :future)))) (:name #("Stuck" 0 5 (face org-ql-view-heading-2)) :where (stuck)) (:name #("UNDERWAY" 0 8 (face org-ql-view-heading-2)) :where (todo "UNDERWAY" "NOW")) (:name #("NEXT" 0 4 (face org-ql-view-heading-2)) :where (todo "NEXT")) (:name #("Neglected" 0 9 (face org-ql-view-heading-2)) :where (and (todo "PROJECT") (not (descendants (and (todo "WAITING") (scheduled)))) (not (or (clocked :from -7) (descendants (clocked :from -7)))) (not (stuck))) :group nil)) :append nil :narrow nil)
  (progn (setq section-name (or section-name "[unnamed section]")) (add-face-text-property 0 (length section-name) 'org-ql-view-heading-1 nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from (or section-from from) :where (or section-where where) :sort (or section-sort sort) :group (or section-group group) :queries (or section-queries queries) :append append :narrow section-narrow) (setq append t))
  (let ((section-name x295) (section-from x296) (section-where x297) (section-sort x298) (section-group x299) (section-queries x300) (section-narrow x301)) (progn (setq section-name (or section-name "[unnamed section]")) (add-face-text-property 0 (length section-name) 'org-ql-view-heading-1 nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from (or section-from from) :where (or section-where where) :sort (or section-sort sort) :group (or section-group group) :queries (or section-queries queries) :append append :narrow section-narrow) (setq append t)))
  (let* ((x295 (map-elt x294 :name)) (x296 (map-elt x294 :from)) (x297 (map-elt x294 :where)) (x298 (map-elt x294 :sort)) (x299 (map-elt x294 :group)) (x300 (map-elt x294 :queries)) (x301 (map-elt x294 :narrow))) (let ((section-name x295) (section-from x296) (section-where x297) (section-sort x298) (section-group x299) (section-queries x300) (section-narrow x301)) (progn (setq section-name (or section-name "[unnamed section]")) (add-face-text-property 0 (length section-name) 'org-ql-view-heading-1 nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from (or section-from from) :where (or section-where where) :sort (or section-sort sort) :group (or section-group group) :queries (or section-queries queries) :append append :narrow section-narrow) (setq append t))))
  (progn (ignore (mapp x294)) (let* ((x295 (map-elt x294 :name)) (x296 (map-elt x294 :from)) (x297 (map-elt x294 :where)) (x298 (map-elt x294 :sort)) (x299 (map-elt x294 :group)) (x300 (map-elt x294 :queries)) (x301 (map-elt x294 :narrow))) (let ((section-name x295) (section-from x296) (section-where x297) (section-sort x298) (section-group x299) (section-queries x300) (section-narrow x301)) (progn (setq section-name (or section-name "[unnamed section]")) (add-face-text-property 0 (length section-name) 'org-ql-view-heading-1 nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from (or section-from from) :where (or section-where where) :sort (or section-sort sort) :group (or section-group group) :queries (or section-queries queries) :append append :narrow section-narrow) (setq append t)))))
  (let ((x294 (car tail))) (progn (ignore (mapp x294)) (let* ((x295 (map-elt x294 :name)) (x296 (map-elt x294 :from)) (x297 (map-elt x294 :where)) (x298 (map-elt x294 :sort)) (x299 (map-elt x294 :group)) (x300 (map-elt x294 :queries)) (x301 (map-elt x294 :narrow))) (let ((section-name x295) (section-from x296) (section-where x297) (section-sort x298) (section-group x299) (section-queries x300) (section-narrow x301)) (progn (setq section-name (or section-name "[unnamed section]")) (add-face-text-property 0 (length section-name) 'org-ql-view-heading-1 nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from (or section-from from) :where (or section-where where) :sort (or section-sort sort) :group (or section-group group) :queries (or section-queries queries) :append append :narrow section-narrow) (setq append t))))) (setq tail (cdr tail)))
  (while tail (let ((x294 (car tail))) (progn (ignore (mapp x294)) (let* ((x295 (map-elt x294 :name)) (x296 (map-elt x294 :from)) (x297 (map-elt x294 :where)) (x298 (map-elt x294 :sort)) (x299 (map-elt x294 :group)) (x300 (map-elt x294 :queries)) (x301 (map-elt x294 :narrow))) (let ((section-name x295) (section-from x296) (section-where x297) (section-sort x298) (section-group x299) (section-queries x300) (section-narrow x301)) (progn (setq section-name (or section-name "[unnamed section]")) (add-face-text-property 0 (length section-name) 'org-ql-view-heading-1 nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from (or section-from from) :where (or section-where where) :sort (or section-sort sort) :group (or section-group group) :queries (or section-queries queries) :append append :narrow section-narrow) (setq append t))))) (setq tail (cdr tail))))
  (let ((tail sections)) (while tail (let ((x294 (car tail))) (progn (ignore (mapp x294)) (let* ((x295 (map-elt x294 :name)) (x296 (map-elt x294 :from)) (x297 (map-elt x294 :where)) (x298 (map-elt x294 :sort)) (x299 (map-elt x294 :group)) (x300 (map-elt x294 :queries)) (x301 (map-elt x294 :narrow))) (let ((section-name x295) (section-from x296) (section-where x297) (section-sort x298) (section-group x299) (section-queries x300) (section-narrow x301)) (progn (setq section-name ...) (add-face-text-property 0 ... ... nil section-name) (taxy-org-ql-view :buffer buffer :columns columns :name section-name :from ... :where ... :sort ... :group ... :queries ... :append append :narrow section-narrow) (setq append t))))) (setq tail (cdr tail)))))
  (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '(:buffer :queries :from :where :sort :group :columns :sections :allow-other-keys)) (if (cdr --cl-keys--) nil (error "Missing argument for %s" (car --cl-keys--))) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ... --cl-rest--))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:buffer :queries :..." (car --cl-keys--)))))) (let ((tail sections)) (while tail (let ((x294 (car tail))) (progn (ignore (mapp x294)) (let* ((x295 ...) (x296 ...) (x297 ...) (x298 ...) (x299 ...) (x300 ...) (x301 ...)) (let (... ... ... ... ... ... ...) (progn ... ... ... ...)))) (setq tail (cdr tail))))))
  (let* ((buffer (car (cdr (plist-member --cl-rest-- ':buffer)))) (queries (car (cdr (plist-member --cl-rest-- ':queries)))) (from (car (cdr (plist-member --cl-rest-- ':from)))) (where (car (cdr (plist-member --cl-rest-- ':where)))) (sort (car (cdr (plist-member --cl-rest-- ':sort)))) (group (car (cdr (plist-member --cl-rest-- ':group)))) (columns (car (cdr (plist-member --cl-rest-- ':columns)))) (sections (car (cdr (plist-member --cl-rest-- ':sections)))) (append nil)) (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '...) (if (cdr --cl-keys--) nil (error "Missing argument for %s" ...)) (setq --cl-keys-- (cdr ...))) ((car (cdr ...)) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:buffer :queries :..." (car --cl-keys--)))))) (let ((tail sections)) (while tail (let ((x294 (car tail))) (progn (ignore (mapp x294)) (let* (... ... ... ... ... ... ...) (let ... ...))) (setq tail (cdr tail)))))))
  taxy-org-ql-report(:buffer "Clients" :queries ((:name #("Agenda" 0 6 (face org-ql-view-heading-2)) :where (and (or (scheduled) (deadline auto)) (not (done))) :group (((planned :today :name "Today")) ((planned :past)) ((planned :future)))) (:name #("Stuck" 0 5 (face org-ql-view-heading-2)) :where (stuck)) (:name #("UNDERWAY" 0 8 (face org-ql-view-heading-2)) :where (todo "UNDERWAY" "NOW")) (:name #("NEXT" 0 4 (face org-ql-view-heading-2)) :where (todo "NEXT")) (:name #("Neglected" 0 9 (face org-ql-view-heading-2)) :where (and (todo "PROJECT") (not (descendants (and (todo "WAITING") (scheduled)))) (not (or (clocked :from -7) (descendants (clocked :from -7)))) (not (stuck))) :group nil)) :sort (priority date reverse) :group (priority) :columns ("Keyword" "Heading\\Parent" "Pri" "Planning" "Tags") :sections ((:name #("Client 1" 0 8 (face org-ql-view-heading-1)) :from "~/client1.org") (:name "Client 2" :from "~/client2.org")))
  eval-buffer()  ; Reading at buffer position 1301
  funcall-interactively(eval-buffer)
  command-execute(eval-buffer record)
  execute-extended-command(nil "eval-buffer" nil)
  funcall-interactively(execute-extended-command nil "eval-buffer" nil)
  command-execute(execute-extended-command)

[alphapapa]

@ParetoOptimalDev Thanks, I hadn't tried it on empty results yet. :)

[ParetoOptimalDev]

I didn't want to try it on empty results, I just had issues. Maybe because I wasn't guessing the format of ~/client1.org correctly?

Can you give a sample org file you know works with the instructions above? I'm very interested in trying this out more for my workflow.

Also, did you notice how the flexibility and feel of the agenda when adding taxy is approaching what it's like to just work from the org file? Is there some deeper meaning or lesson here? lol

[anpandey]

If you use Nix here's an expression for trying out the branch (you'll probably have to run emacs -q since it doesn't load any other packages:

{ pkgs ? import <nixpkgs> {} }:
let
  org-taxy-emacs = (pkgs.emacsWithPackages (epkgs:
    [
      (epkgs.trivialBuild rec {
        pname = "org-ql";
        version = "git";
        src = pkgs.fetchFromGitHub {
          owner = "alphapapa";
          repo = "org-ql";
          rev = "wip/taxy-org-ql-view";
          sha256 = "sha256-HM5aOi2WKD6LARlUE3FmR2MHlVEaItWuRxSkAE+T5dw=";
        };
        propagatedUserEnvPkgs = with epkgs.melpaPackages; [
          dash ts f ov
          org-super-agenda
          epkgs.elpaPackages.peg
          helm helm-org
          epkgs.taxy epkgs.taxy-magit-section
        ];
        buildInputs = propagatedUserEnvPkgs;
      })
    ]
  ));
in pkgs.stdenv.mkDerivation {
  name = "test-env";
  buildInputs = with pkgs; [
    org-taxy-emacs
  ];
  src = null;
}

[alphapapa]

@anpandey As much as I admire Nix for what it is, that code makes me appreciate Quelpa even more. :) Anyway, thanks. I still intend to merge this branch someday, just haven't had much time for it lately.

[hpfr]

Hi, I’m interested in trying this but wanted to check if anything has changed in the past few months. Is wip/taxy-org-ql-view still the suggested branch?

[alphapapa]

Hi, I’m interested in trying this but wanted to check if anything has changed in the past few months. Is wip/taxy-org-ql-view still the suggested branch?

If I pushed anything notable to it, I would have mentioned it here. It should be as testable now as it was then.

[mskorzhinskiy]

Very nice work! Looking forward for getting it merged some time in the future. I have played with it a little and below are my findings. Note, however, that I have not dived deep into the code to figure out how to work with the new system, just used mostly intuition.

• It seems no way not to use :sections? I have tried to this:

  (taxy-org-ql-report
  :buffer "Blah"
  :queries '((:name "test"
            :where (not (done))))
  :sort '(priority date reverse)
  :group '(priority)
  :columns '("Keyword" "Heading\\Parent" "Pri" "Planning" "Tags")
  :from (org-agenda-files))

And this just returned me nil.

• Separations between section and queries is a little bit confusing. I see the point, but for me, I would like just run queries A, B and C on the whole of my agenda files. Possibly :from should be part of individual query?

• Groupping with tags seems to be not working with I suspect, inherited tags Query:

  (taxy-org-ql-report
  :buffer "Blah"
  :queries '((:name "Today"
            :where (or (and (todo "TODO")            ;
                            (or (priority "A")
                                (scheduled :to today)
                                (ancestors
                                 (priority "A" "B"))))
                       (and (deadline :auto)
                            (not (done)))
                       (and (planning :from today :to +1)
                            (not (done)))
                       (and (todo "DONE")
                            (closed :on today))))
           (:name "Dashboard"
            :where (or (and (not (done))
                            (or (planning :to 7 :from -1)
                                (deadline auto)
                                (and (todo)
                                     (not (ancestors (todo "WAIT")))
                                     (not (descendants (todo)))
                                     (not (ts-active)))))
                       (tags-inherited "refile")
                       (and (todo "DONE")
                            (closed :to today :from -14)))))
  :sort '(priority date reverse)
  :group '(tags)
  :columns '("Keyword" "Heading\\Parent" "Pri" "Planning" "Tags")
  :from (org-agenda-files)
  :sections `((:name "haha"
              :from ,(org-agenda-files))))

  Start of backtrace: (I can send the whole backtrace if you interested personally)

  Debugger entered--Lisp error: (wrong-type-argument buffer-or-string-p (#("legal" 0 5 (inherited t))))
  add-face-text-property(0 1 org-ql-view-heading t (#("legal" 0 5 (inherited t))))
  ...

• Global and per-query grouping doesn't play along, it seems per-query will overwrite global? (but probably the most sane variant yes)

• Heading grouping seems to be doing... nothing?

• Columns are very nice! However I personally prefer instead of strings to have symbols so that I can 'goto' definition quickly to see documentation which each column means exactly

• Would be really nice to have more things to have faces for, to apply more personal styling. E.g., in 'heading\Parent' parent is in italic, which makes columns to be misaligned in my configuration

• Would it be possible to define my own title instead of "*Taxy Org QL View: MY TITLE"? And instead just "MY TYTLE"?

• Would it be possible to create lower-level groups from high level grouping? E.g., if I had this org-mode hierarchy:

• One Two Zero Two One :sport: Something Something ** Two Two *** Three :sport:

And get this view, by getting everything tagged as sport?

One
  Two One
    Something
    Something
  Two Two
    Three

Like get kind of sparse tree inside QL view?

• There are some parts of taxy elisp file written with tabs as indentation ;-)

[alphapapa]

@mskorzhinskiy I appreciate your trying it out and giving feedback, but that is...a lot of feedback and many questions. What needs to happen next is for me to rebase this branch on master again, then spend some more time using it and making the API more consistent (e.g. the "sections" and "queries" probably need to be consolidated or clarified). Then I need to get it into an nearly bug-free state, and then merge it as a separate library that users can choose to experiment with as it develops further. Maybe in the version after that it can be added to the documentation so more users can begin using it, and maybe in the version after that it can be considered a "first-class," supported feature. And eventually maybe it can replace org-ql-view and even take on that name.

Please note that I have no timeframe for this plan, as I have limited time to work on this.

[mskorzhinskiy]

No expectation this to be merged any time soon, tried it because it seemed very interesting to try. You can tag me in this issue once again when you would be interested in the detailed feedback, I would be willing to try it once more.

[ParetoOptimalDev]

When I get this working locally my ideal would be ability to sort arbitrary groups based on their total priority.

For instance:

* work
*** TODO project 1 [0/1]
**** NEXT [#C] do extra non-urgent code review
* household
** TODO redesign kitchen [0/1]
**** NEXT [#A] choose new tiles
* finances
** [#A] TODO taxes
*** NEXT [#A] audit last quarter spending

Org agenda would look like:

finances
   taxes
      NEXT [#A] audit last quarter spending
household
   redesign kitchen
      NEXT [#A] choose new tiles
Work
    project 1
       NEXT  [#C] do extra non-urgent code review

Note that finances "category" here sorts as higher priority because it itself has [#A] priority as well as one of it's subtasks.

[ParetoOptimalDev]

I've got this working on my laptop and find it both very useful, but seems to require a bit of a shift in perspective where files are their own groups. I believe I'll likely come to want to have at least groups of files for each of these.

Unsurprisingly given this is unreleased, experimental, and something you are just doing as you have free time.

With that said, I found a couple of bugs:

• Per query grouping is not respected. For example neglected uses the global :group '(priority) rather than :group nil
• Same as above, but for sorting

I also noticed that there isn't yet support for ordering of sections based upon the priority within that group. For me, having the ability to group priorities like this is very valuable, but being able to float up the one with more high priority items to the top would make it even more valuable.

For example if I had 3 high priority household tasks, and 1 high priority errand, I'd like to move the household section above the errand section regardless of the order of the queries.

Perhaps others might find this idea exciting or useful.