":scheduled today" doesn't show the items in the correct (Today) group on days other than today

[berkan]

Hello,

Thanks for org-super-agenda, it's a big help.

As the title says, I have an expectation problem with the way ":scheduled today" works.

It works as expected for the day that is actually today. In other words, items scheduled for today appear in the correct group.

But let's say I have items scheduled to be done tomorrow. When I look at / go forward to tomorrow's day-view/-block, I expect those items to appear in that day's ":scheduled today" group. But they do not -- instead, they will be under eg "Other items".

Is the way I expect this to behave not the more natural way? Or am I missing an obvious point?

Here's an org file to demonstrate -- just save it and follow the TODOs :)

Many thanks Berkan

* Tasks

** TODO C-xC-e the sexp to schedule this task for today  :task1:
   SCHEDULED: <2018-07-05 Thu>

   #+BEGIN_SRC elisp
     (org-schedule nil ".")
   #+END_SRC

** TODO C-xC-e the sexp to schedule this task for tomorrow  :task2:
   SCHEDULED: <2018-07-06 Fri>

   #+BEGIN_SRC elisp
     (org-schedule nil "+")
   #+END_SRC

** TODO C-xC-e the sexp to display the agenda

   #+BEGIN_SRC elisp
     (let ((org-agenda-sticky nil)
           (org-agenda-span 2) ; show 2 days
           (org-agenda-start-day ".") ; start from today
           (org-agenda-start-on-weekday nil)
           (org-super-agenda-groups '((:name "Today" :scheduled today))))
       (org-super-agenda-mode)
       (save-excursion
         (goto-char (org-find-olp (list "Tasks") 't))
         (org-agenda nil "a" 'subtree)))
   #+END_SRC

   Expected: The first task will appear under "Today" in today's block.
   Actual: As expected.

   Expected: The second task will appear under "Today" in tomorrow's block.
   Actual: The second task appears under "Other items" in tomorrow's block.

[alphapapa]

Hi,

Thanks for org-super-agenda, it's a big help.

Glad to hear it, thanks.

Is the way I expect this to behave not the more natural way? Or am I missing an obvious point?

Unfortunately org-super-agenda isn't really designed to work with multiple agenda blocks or multi-day spans. The way it works is to post-process all of the agenda items at once, put them into groups, and then hand them back to the agenda finalizing functions. It has no concept of nor relation to what spans or blocks the rest of the Org agenda code produces. So you could say that the date-related grouping functions are designed to work with single-day agenda views.

I guess I should put that in the readme, although it's not that easy to explain clearly. Suggestions for how to word it are welcome. :)

[alphapapa]

P.S. Although it would be nice to fix this, I don't think it's practical at all. The Org Agenda code is very complicated and not very modular. We would need to either reimplement large parts of the agenda ourselves, or insert a bunch of calls to intercept and modify agenda items at other stages of item collection, which would be extremely complicated, error-prone, and very unlikely to be merged into Org proper. (Not to mention that I have no interest in trying to do so, haha. ;)

However, if you have any interest in hacking on the Org Agenda, you might find this interesting: https://github.com/alphapapa/org-agenda-ng It's a very primitive agenda by comparison, but it has a more modular design that might be a foundation or a prototype for rewriting the Org Agenda completely, perhaps with features like grouping built-in.

[berkan]

Hi again, thanks for the explanation.

Just to be clear: Does the difficulty/impracticality of a solution relate to getting it to work with multi-day-spans; or to making the date that ":scheduled today" refers to relative to the active day-view, even on single-day views?

The issue is I was trying to highlight is, "today" always seems to mean the actual today, no matter which day-view we're looking at. On today's single-day-view, it's fine. But on another day's single-day-view, it still pivots on the actual today, so the tasks there are grouped based on today's date and not the date of that day's view.

(For instance, if you take out the org-agenda-span line from the above example, it still behaves as I described)

I actually would be happy if it worked (as I expect :) with only single-day agenda views - I rarely use day-spans.

org-agenda-ng looks very interesting, I will try it as well. (But I wonder if the behaviour of "scheduled :today" will be different).

[alphapapa]

The issue is I was trying to highlight is, "today" always seems to mean the actual today, no matter which day-view we're looking at. On today's single-day-view, it's fine. But on another day's single-day-view, it still pivots on the actual today, so the tasks there are grouped based on today's date and not the date of that day's view.

Okay, now I understand what you mean...

Just to be clear: Does the difficulty/impracticality of a solution relate to getting it to work with multi-day-spans; or to making the date that ":scheduled today" refers to relative to the active day-view, even on single-day views?

I think these are separate issues, and from what I can tell, neither one would be simple to fix. I thought that org-today accounted for the, shall we say, "perspective date," in agenda views, but it doesn't seem to. It does account for org-extend-today-until, but that also apparently does not set the perspective date when gathering agenda items (e.g. see org-agenda-get-scheduled). That seems to be set by the date variable in e.g. org-agenda-get-day-entries. (org-agenda.el does not use lexical binding, so date in org-agenda-get-scheduled is bound by the function that calls it.) It might be possible to access that variable in org-super-agenda--group-scheduled, but I almost don't even want to try, because the name date is so ambiguous in this context, haha. But that might be possible, and that might fix the single-day-span views of days other than the real-world today.

org-agenda-ng looks very interesting, I will try it as well. (But I wonder if the behaviour of "scheduled :today" will be different).

It won't be different, because this issue is within org-super-agenda. What I meant was, it might be possible to build a more flexible Org Agenda system that would make it easier to fix these issues in org-super-agenda. The problem is basically that we don't have access from org-super-agenda to the date that the Org Agenda is building a view for (the "perspective date", as I call it), so we don't know whether an item is scheduled on that date; we only know the current real-world date and the date of the item.

So a more flexible underlying agenda system might e.g. set a defvar like org-agenda-perspective-date while building a view for that date, and then we could easily know which date to compare items to.

And, in fact, that might be a feasible solution to this problem as well: just do that in the existing Agenda code, which would be only a few lines of code, and might be accepted in Org proper. However, it would likely be not quite as simple as it seems, because it would probably need to be set in several different places, in different functions, ensuring that it is done properly and in every place so that it's consistent. But it's probably worth asking about on the mailing list.

[alphapapa]

Note, I think I was confused: setting a special variable wouldn't do any good. But setting an agenda-perspective-date text property on each item probably would let us fix this, at least theoretically.

[codygman]

I think I may have experienced this. Given org file:

* TODO thing 1   :foo:
* TODO thing 2   :foo:
* TODO thing 3   :bar:

And groups:

(setq org-super-agenda-groups
      '((:name "foo" :tag "foo")
    (:name "superset of foo" :tag ("foo" "bar"))))

I'd like the agenda to look like:

foo
====
TODO thing 1
TODO thing 2

superset of foo
==================
TODO thing 1
TODO thing 2
TODO thing 3

Instead since things matched in foo seem to be discarded I see:

foo
====
TODO thing 1
TODO thing 2

superset of foo
==================
TODO thing 3

The real world use of this being something like having a tag you want at the top of your agenda like "Top3PrioritiesToday", but also wanting your time grid to have those items if you happen to be scrolling through that section.

[alphapapa]

@codygman Please refer to the documentation. Grouping selectors consume items.

[nchachereau]

@alphapapa : the variable org-agenda-current-date looks like what you were looking for (it's been two years, after all). I think this would make the whole thing easier (I would guess we rarely want "today" and usually want what you called the "perspective date" of the agenda view).

[alphapapa]

@nchachereau Thanks, I'll take a look at that.

[real-or-random]

Until that problem is solved properly, here's a cheap hack if all you need is selecting items scheduled on or with deadline on the "perspective date".

  (defun tim/is-today-deadline-entry-p (item)
    (string-prefix-p "  Deadline:" item))
  (defun tim/is-today-scheduled-entry-p (item)
    (string-prefix-p "  Scheduled:" item))

(inspired by #238)

Then you can select with, e.g., :pred tim/is-today-deadline-entry-p, which simply matches all lines starting with "Deadline:".

[alphapapa]

Looking at this again, changing the date-based selectors to use org-agenda-current-date would not be a trivial change. It's past time to tag a new v1.3 release, so I'll revisit this in the future.