Automatically append Task completed date

[justyns]

This adds an option (off by default) that appends todays date to a task when it is marked as completed, and looks like this:

It also shows up in queries so you should be able to filter on it, like task where page = @page.name and completed < "2024-02-16"

I mentioned this on discord a little, but I briefly considered creating a new plug for this. However, I also wanted the ability to extract the completed date from a task in queries. I couldn't figure out how to parse that and add the 'completed' key with a separate plug since some of the related code isn't part of the tasks plug.

[zefhemel]

This looks very nice. I'll merge it once I'm able to test it (in a week or so).

One minor concern I have, not with this change specifically but with adding more and more specific syntax additions is the scalability. There is also https://silverbullet.md/Attributes that you can add to tasks and give you some of this functionality for free (like the indexing part). The only drawback of using that instead is the looks. [completed: 2024-02-15] looks a tad less nice than the emoji version...

The reason I hardcoded the deadline support was that attributes didn't yet exist. Now they do. The question is: should we not switch to that?

Any opinions?

[NeilHanlon]

I think yes, personally.

I would also say a "Wait" attribute would be good, I use this a lot to add a task I don't want to see until a certain day (ie next week, next month, whatever)

[devzero]

I also have a waiting: string attribute I use as well as an attribute for numerical priority for prioritized tasks. These are both present in obsidian tasks

On Fri, Feb 16, 2024, at 9:10 AM, Neil Hanlon wrote:

I think yes, personally.

I would also say a "Wait" attribute would be good, I use this a lot to add a task I don't want to see until a certain day (ie next week, next month, whatever)

— Reply to this email directly, view it on GitHub https://github.com/silverbulletmd/silverbullet/pull/715#issuecomment-1948453433, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACM6HRRUVNZ6O3X7HOLXL3YT5SFDAVCNFSM6AAAAABDLZP7SWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBYGQ2TGNBTGM. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[justyns]

I do like the generic approach better than hardcoding deadline/completed/etc, but agree the attributes wouldn't look as nice. After thinking a bit, I had two ideas:

1) Add a new syntax for attributes so that 📅 <date> gets parsed as [deadline: <date>]. In SETTINGS, we could have something like

attributeMap:
  📅: deadline
  ✅: completed
  ⌛: scheduled

2) Add a way to style attributes so that [deadline: <date>] gets rendered in the live preview as 📅 <date>. I'm not sure if it could be done entirely with css or not, it might also need some sort of map in the settings.

As a side-note, I couldn't find a good standard/convention of adding extra attributes to tasks in markdown other than the Obsidian Tasks plugin. I feel like taking inspiration from it should be safe for compatibility purposes.

[Maarrk]

What if we make Unicode valid anywhere and [📆: <date>] is both the same universal attribute syntax and almost as concise as the special hardcoded case. Then you'd also use same emoji in queries, instead of the word deadline.

Also, you can already make an alias for the picker from:deadline to 📆

As concise without introducing aliases, lets you get more kinds of dates in the same way (like "wait" mentioned above), introduces users to attributes, and to me is simpler to explain (no magic values)

Coming back to this PR, that could be expressed in SETTINGS:

tasks:
  completedDateAttribute: ✅

But if you really dislike emoji you can change it to completed, and if it's missing, it's the default off

---

Side effect for more Unicode: I once had the issue that I had to translate my tags to English, because Polish word for book didn't work here:

```query
książka where ...

Disadvantage of that: you can't just write `[A-Za-z]` regex everywhere

[devzero]

Or better yet, a more generic mapping of a regex that could include an emoji and have a match that gets converted into an output string which could be an attribute before parsing and processing of the task

On Fri, Feb 16, 2024, at 6:38 PM, Marek S. Łukasiewicz wrote:

What if we make Unicode valid anywhere and [📆: <date>] is both the same universal attribute syntax and almost as concise as the special hardcoded case. Then you'd also use same emoji in queries, instead of the word deadline.

As concise without introducing aliases, lets you get more kinds of dates in the same way (like "wait" mentioned above), introduces users to attributes (my dad has this important or not, urgent or not system), and to me is simpler to explain (no magic values)

Coming back to this PR, that could be expressed in SETTINGS:

tasks: completedDateAttribute: ✅ But if you really dislike emoji you can change it to completed, and if it's missing, it's the default off

— Reply to this email directly, view it on GitHub https://github.com/silverbulletmd/silverbullet/pull/715#issuecomment-1949486941, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACM6HQH4FXJIT6YCDP6BZDYT7UXHAVCNFSM6AAAAABDLZP7SWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBZGQ4DMOJUGE. You are receiving this because you commented.Message ID: @.***>

[justyns]

For queries, I'd much rather be able to type 'deadline' or 'scheduled' instead of 📆 and ⌛ . I guess it doesn't make much of a difference with autocomplete, but personal preference?

I've been playing around with this, and I think we could do @devzero 's regex idea without too many changes.

In SETTINGS, we'd have something like: ❗

regexAttributes:
  "\\s⌛\\s+(\\d{4}-\\d{2}-\\d{2})": "scheduled"
  "\\s📆\\s+(\\d{4}-\\d{2}-\\d{2})": "deadline"
  "\\s❗\\s+(\\d)": "priority"

This would map ⌛ 2024-05-15 to task.scheduled = "2024-05-15" and would be available in queries/etc. It'd also mean the task itself could have either ⌛ 2024-05-15 or [scheduled: 2024-05-15] and both formats would work. I think this would be good for helping users who are migrating from Obsidian too.

I have a POC of this working locally, but only for tasks currently. I'm not sure if it'd also be useful elsewhere?

[Maarrk]

For queries, I'd much rather be able to type 'deadline' or 'scheduled' instead of 📆 and ⌛ . I guess it doesn't make much of a difference with autocomplete, but personal preference?

It feels like people believe it's a sin, to put anything non-ASCII in code 😅.

But now you're making everyone write regex in SETTINGS, that's really hard jump for users who don't program. And this one is with capturing group, so another special case

What if emoji aliases worked both for the picker and names of attributes in queries? A way to get all described above but simpler config without regex. And I really want to make the more universal Attribute syntax organically discoverable through this cool feature. So [❗: 2] into task.priority = 2 if you have SETTINGS

emoji:
  aliases:
    priority: ❗

[devzero]

It's not like there aren't already regex all over the slash commands. Isn't this "the hacker's PKM"? Additionally, the point of the emojis is to reduce data entry and improve readability and compatibility with existing notes people may already have. Allowing arbitrary control will allow lots of options in the future that haven't been planned for even without writing any more code.

[Maarrk]

TL;DR: Y'all are right in practice

It's not like there aren't already regex all over the slash commands. Isn't this "the hacker's PKM"?

Looking at other people I host for, you can get a lot done without any coding, both settings and templates are easier to understand than regex, and I wouldn't give up on that.

Additionally, the point of the emojis is to reduce data entry

OK, I agree, that the "one Attribute syntax to rule them all" is an elegant concept, but those extra [ : ] are a real pain on mobile after some thought.

... and improve readability and compatibility with existing notes people may already have

I didn't think about compatibility, since I didn't come from Obsidian before. I'd be afraid of making "Obsidian vol. 2: the Open Source One". But if that's a majority of users, then improving compatibility is going to have the biggest UX impact across the userbase.

[justyns]

It feels like people believe it's a sin, to put anything non-ASCII in code 😅. But now you're making everyone write regex in SETTINGS, that's really hard jump for users who don't program

lol that's a good point 😅

One benefit to regex is that we wouldn't be limited to <emoji> <value> but could easily add other syntaxes too. I'm struggling to think of a good example though, so it may be overkill. Data validation would also be a small bonus.

What do y'all think about offering two different methods?

emojiAttributes:
  📅: deadline
  ✅: completed
  ⌛: scheduled
  ❗: priority

regexAttributes:
  "\\s⌛\\s+(\\d{4}-\\d{2}-\\d{2})": "scheduled"
  "\\s📆\\s+(\\d{4}-\\d{2}-\\d{2})": "deadline"
  "\\s✅\\s+(\\d{4}-\\d{2}-\\d{2})": "completed"
  "\\s❗\\s+(\\d)": "priority"

These would be almost identical in practice, and I'd imagine the javascript part of it would just convert emojiAttributes into regexAttributes.

Something else I was considering is if there's a way to change this from SETTINGS to space script or plugs to allow plugins to add their own attributes. It'd be the opposite of simplifying things for users, but maybe would be more extensible? Something like:

silverbullet.registerAttribute({
  name: "deadline",
  pattern: "\\s📅\\s+(\\d{4}-\\d{2}-\\d{2})",
  transform: (match) => {
    return {
      attributeName: "deadline",
      value: match[1],
    };
  }
});

// And a shortcut function that uses the above but just maps an emoji
silverbullet.registerEmojiAttribute({
  emoji: "✅",
  attributeName: "completed"
});

re: compatibility I don't currently use Obsidian (or any markdown-based task manager) so I'm not super concerned personally, but it seems like there is a big user base who use the Obsidian Tasks syntax. I also couldn't find very many other conventions/examples that made more sense.

[zefhemel]

Excellent discussion! It really helped me figure some things out. Here’s my current thinking.

What I’d really like to be possible is to move a lot of task related functionality to libraries (as in: I’d like there to be a Library/Tasks or whatever that implements things that you can just import). To an extent this can already be done, task related commands can be migrated to space script commands. Another step is what’s discussed here: how to specify task attributes.

At the lowest level this is already possible with the [attribute: value] syntax, but I agree it would be nice to offer more custom syntax. I would not hardcode this to be emoji relate only, rather I’d use this as an opportunity to make “attribute extraction” a more general thing, not just for tasks but for any type of object you can add attributes to already (so pages, paragraphs and items as well). Luckily there’s a single point in the code base where attributes are extracted that can be extended: https://github.com/silverbulletmd/silverbullet/blob/main/plug-api/lib/attribute.ts#L15

How? I really like @justyns idea of using a type of “attribute extraction” space script function for this, because it would also allow for easy distribution of these things, and it can potentially be made more flexible than just regex-based matching (although that’s likely how you’d implement most of these).

A conceptual API I’d go for would maybe be:

/**
 * - text: would contain the (markdown) text of the thing to extract attributes from (can be a paragraph text, item or task text)
* - tree: same content but as a parse tree if your into that type of thing 
 * - type: would be “page”, “paragraph”, “item”,  “task” depending on the type of thing we're trying to extract attributes from 
 * @return a Record<string, any> mapping of newly extracted attributes, or null if nuthin’
 */
silverbullet.registerAttributeExtractor("task", (text, tree) => {
    // … do regex magic here
    return { completed: completedDate };
});

The implementation challenge here would be that all indexing happens in plug space right now, whereas space script runs in the server/client main thread, so we’d have to add a syscall to invoke space script stuff from inside of the (index) plug.

I realize this kind of escalates this whole “hey let me just add task completion date” PR a bit so happy to take this on when I get back. Or if you want to give it a try, feel free. And of course challenge me on this design, I’m not saying it’s perfect.

Thoughts?

[zefhemel]

By the way: why I see attribute extraction as a more general idea as useful.

This is what I'd like to enable:

On my daily journal page I write:

• Had coffee with [[John]]

Which is slightly more natural than something like:

• Had [beverage: coffee] with [person: John] #coffee-meet

Then I go off and write a 4 line space script that matches the pattern of "Had X with Y", extracts X and Y into attributes and now I can add a query block on my John page (or any person page) that lists references to any type of beverage I've had with them and when (because the daily note reference). Or I could write a query that lists all people I didn't have coffee with for some time so maybe I reach out.

It could be quite powerful.

[justyns]

Thanks @zefhemel , I like the idea of a Library/Tasks and using space script/etc. It should be more flexible in the future.

Had coffee with [[John]]

This sounds like an awesome idea! I could see it being a little tricky when you have variations like (Had|Ate|Drank) (lunch|dinner|coffee|drinks) with Y since [beverage: X] wouldn't cover all of those scenarios. But I think it'd be worth it.

silverbullet.registerAttributeExtractor

The potential syntax for this seems fine to me. If I (or someone else) wants to, we could also wrap it with a helper function with a simpler syntax for things like the emoji-specific attributes.

Additional thoughts:

I brought this up on the discord, but putting it here too. I like the idea of having something similar to silverbullet.registerAttributeExtractor but that turns an attribute like [scheduled: 2024-02-18] into ⌛ 2024-02-18 in the live preview. Not specific to attributes, it could be used for things like replacing John with [[John]] or [[John]] with <johns avatar.png> [[John]]. Or https://jira.company/JIRA-123 with <company logo.png> JIRA-123

Zamuz mentioned collapsing the attributes like [beverage: coffee] to a simpler [coffee] to improve readability. This approach could allow for that too

Re: "hey let me just add task completion date" - I'm totally fine with scrapping this PR if the intent is to move the tasks stuff out of the core code. I do think there would need to be some sort of event that we can hook into, maybe another silverbullet.registerRunOnHook("task.completed", (task) => {};) idea?

[devzero]

you might want to register any task state change instead, as they might want to undo-things if I task gets unchecked or if there are more than two task states.

I really like the idea of registerAttributeExtractor a generic functional transformation on objects that users can register to apply for populating the object store at parsing time. One question about whether it runs on the server or the client, would that break offline mode for pages that use this? Is there a way to run space scripts on the server or the client?

A complementary feature might also be registerITagExtractor, to add "implicit" tags to a page/task/object with a similar mechanism. This could allow you to add inferred tags (like auto-adding parent tags in a tag tree). Or add the #coffee-meet tag if the content of the page matches.

[zefhemel]

Getting there

[paletochen]

Nice looking!

Will it undo the addition of the completed date if the task is marked undone again?

[zefhemel]

No

[zefhemel]

Ok, I just merged registerAttributeExtractor into main (available on Edge soon), the example on the docs page shows how to implement what you do in this PR (more or less) using space script.

[zefhemel]

https://silverbullet.md/Space%20Script#Custom%20attribute%20extractors

[zefhemel]

There's now also a way to listen to events from within space script. All that needs to be added now to the task plug is that it will trigger an event when a task status changes, then you can create a listener for this event that automatically adds the completion date.

[zefhemel]

Hmm this doesn't seem to work in read only mode SB yet.

[paletochen]

I also cannot make it work in my sync_only instance. I don't see any errors in the console though

[zefhemel]

Ok. Will check this tomorrow

[zefhemel]

Ok, both issues should be fixed now and the https://silverbullet.md/Space%20Script now uses adding the task completion task automatically when the task state changes as an example.

[paletochen]

Confirmed it works.

It is really cool because I can now even use the "completed" field to query completed tasks based on specific dates

Thanks!