Date sorting

[stacksjb]

When tasks have a due date, does tod sort by the date itself (oldest to newest)?

Based on my testing, it appears that it sorts based on buckets, but I am getting the newest overdue before the oldest overdue.

[rvbcldud]

I have noticed this as well. This sounds like something I would like to work on.

After some (very) brief investigation, it seems that in the case of task listing all_tasks() is called...

Here is an example of project task listing from projects.rs:

/// All tasks for a project
pub async fn all_tasks(config: &Config, project: &Project) -> Result<String, Error> {
    let tasks = todoist::tasks_for_project(config, project).await?;

    let mut buffer = String::new();
    buffer.push_str(&color::green_string(&format!(
        "Tasks for '{}'",
        project.name
    )));

    for task in tasks::sort_by_datetime(tasks, config) {
        buffer.push('\n');
        buffer.push_str(&task.fmt(config, FormatType::List, false));
    }
    Ok(buffer)
}

It seems to call sort_by_datetime() ... my guess is that it sorts by date/time created instead of due.

Any ideas on how to implement sorting by due date/time?

[stacksjb]

sort_by_datetime() is implemented in tasks.rs on line 480

It looks like 481 is missing the Reverse, which could be the issue? But I'm also finding in my testing it doesn't seem to make a difference, so I think that the Buckets might be overriding my expected behavior.

Let me see if I can find a specific query that works consistently so I can test. Can you give me an example of the exact tasks/situation you are encountering?

@alanvardy it looks like sorting is done on "list process" and "tod task next", but all I can find is by grouping - is it anywhere else? Where is "sort_by_datetime" implemented? When I run "list view" on a project, as far as I can tell, the results are not sorted in any way (they appear to be following the default/manual sort)

One easy way to fix this might be to calculate days overdue and have that as a point that is added (i.e. 1 day overdue = 1 point, sort on that), and then combine and only sort in buckets?

[stacksjb]

@rvbcldud are you running the "list view" and trying to figure out sorting there?

I'm running into this when using "list process" and "tod task next" - I have already customized my weighting to return them as closely to match what i had in Todoist as possible. This is the sorting in the native client I am trying to replicate:

[rvbcldud]

Yes! I am doing tod list view

Let me know if you end up finding a consistent query / any other ideas.

Great work

[stacksjb]

I am using a filter to return tasks tagged as "@Work & (due | overdue | no due date)".

As best as I can tell, the sorting doesn't happen at the granular due date level. Within the code, the sorting happens by score, here:

    /// Determines the numeric value of an task for sorting
    fn value(&self, config: &Config) -> u32 {
        let date_value: u8 = self.date_value(config);
        let priority_value: u8 = self.priority_value(config);

        date_value as u32 + priority_value as u32
    }

date_value is simply the date buckets as described within the config, so as I explained, within "overdue", there isn't differentiation among tasks.

I played around with the code and tested adding a date_skew so that the date_value is skewed by how overdue the task is, and it does, but because it's combined at the end, it means that (at least for my use case) if I have a task 90 days overdue, I have to ensure my priority is more than 90 greater for each level, otherwise the sorting will 'trump' the priority. This then becomes problematic because currently we're using u8, which is limited to 0-255 values.

Is there any way to sort results prior to bucketing? I would think that would solve the issue since then they would be sorted in each 'bucket' before being returned - run sort_by_date first

[stacksjb]

Using s8 instead of u8 may help because then we could use negative priority instead of only positive numbers (allows for negative) - for example, I'd prefer for "no due date" to have a slightly negative priority (we could have "has due date" as a bucket, but that would duplicate and end up with the same result...

[rvbcldud]

within "overdue", there isn't differentiation among tasks.

So, in your experience, does it fail to sort by due date only when they are overdue or in general?

[alanvardy]

Hey guys, just catching up on the conversation here.

• list view doesn't seem to be sorted properly (it is supposed to sort by date), it's some really old code. How do we think it should be sorted now that I need to rewrite it? Oldest first? Or should I use the same "value" method for processing?

• list process sorts by value, I can definitely go from u8 to something larger like i32.

Is there any way to sort results prior to bucketing? I would think that would solve the issue since then they would be sorted in each 'bucket' before being returned - run sort_by_date first

I can definitely do this. Should we just sort by date and then sort by value for all lists of results across the board?

[alanvardy]

Issue for borked date sorting: https://github.com/alanvardy/tod/issues/800

[stacksjb]

I will admit that the weight-based value scoring is powerful but unintuitive and difficult for a new user. I would prefer to define sorting within the code and then let the user select how to sort (or specify as a command line parameter) from the predefined sorting methods.

@alanvardy within the current system is the Struct being sorted (are objects returned in order), or is each object being returned based on the sort by value query?

What I would like to see is two sorting options, maybe "sort_group1" and then "sort_group2", that mirror the official client (Todoist has it "Group By" and then "Sort By", with ascending or descending for each), so you can do something like "Get all P1, sorted by date due, then no due date - then P2, same sorting", and so forth - and then those values being used universally across the board (unless there is the option to override it on the command line).

In code this would mean predefined sorting methods that we then just specify in the config, for example:

"sort1" : "priority",
"sort2" : "date_created",

With the appropriate comparators defined somewhere else in code.

I tried expanding the current sorting to include some sort of date parameter (I tried this on my date_skew branch), but this becomes limited because it requires sorting everything by a value which means that we are essentially assigning every single task object a score which is then exponentially bigger depending on how many tasks you have returned - Todoist limits to 300 tasks per project (on paid plans), but with filters we could feasibly be working through thousands of tasks.

[stacksjb]

Based on my review of the code, it looks like there is just a simple "sort by key", i.e. Integer-based value sort, is that correct? I did not see any Comparator code.

For reference: https://docs.rs/compare/latest/compare/ - I should just be able to stack Comparators.

I think I could work on writing a custom sort, and then just call that comparator to sort the results.

@alanvardy what order are you currently sorting on in your config? In current struct we have: Content (task name) Priority Labels project ID due (due date)

It looks like in addition to int/order based, we would need to add calculations for today, recurring, overdue.

[stacksjb]

@alanvardy if you will take the work of fixing date sort, I'll take the assignment of writing the sort/comparator code for custom sorting.

[rvbcldud]

Looking forward to seeing these new changes! Let me know if there is anything else I can help on with this.

[stacksjb]

@rvbcldud what order would you like to see tasks sorted in?

[rvbcldud]

Ascending order in date, taking into account priority

For example:

• Write email (due yesterday) p2
• Take out trash (due yesterday) p4
• Do laundry (due today) p1
• Eat breakfast (due today) p3 Etc...

Provided there are no overdue tasks, I'd like to see my highest priority task due that day first.

Let me know you what you think

[stacksjb]

Thx.

For me I prefer priority, then due date/time - so all P1 before P2 before P3 before P4, for example:

1. File Timesheet (due today, p1)
2. Email boss (not due, p1)
3. Contact company (p2, due yesterday)
4. Contact other company (p2, due today)
5. Call someone (p2, no due date)
6. Email friend (p3, no due date)
7. Contact person (p4, due friday)
8. Call someone (p4, no due date)

I will ensure both are covered!

[alanvardy]

How would you two deal with a p1 task that is due tomorrow? Would it be first or last?

For myself, I would hope to see a p2 task due today before a p1 task due tomorrow. But I would probably rather see a p1 task due today before an overdue p2 task

My point is that priority is probably more important when dealing with overdue/today tasks And date is probably more important when dealing with tasks in the future. If I have a p1 task due in December, it shouldn't come ahead of a p2 task due today right?

Hopefully this is thought provoking.

[stacksjb]

For me, a p1 task is inherently more important than a P2 task - I do not have a lot of P1 tasks, but they are the ones that are of a core importance. So, even if a P2 task is due, if I don't get my P1 tasks done, then I am not spending time on the most valuable items.

That said, I follow the GTD philosophy and only use due dates on tasks that either must happen by a certain date, or as required for recurring tasks.

I do like adding logic around recurring versus not recurring, because I would probably rather have a not recurring task before a recurring task - so we can certainly add more buckets beyond what exists within the task itself (for example, I like the logic of "now" for due in the next 4 hours as more important)

Either way, I want to ensure that I make the sorting logic flexible enough that people can rearrange it according to what they plan. The goal would be to allow someone to specify a sort order in the config - something like a JSON field, for example:

["now","overdue", "priority", "due_date"];

Anything remaining would default to a default sort order.

I also think I could add logic for due dates (with no time) to be due at a certain time.

[rvbcldud]

That is a great question, Alan!

My preferred order outlined above addresses your concern about a higher priority task "starving" a more urgent lower priority task.

You bring up an interesting point regarding overdue tasks though. I would agree on your order proposed. Ideally overdue tasks are rescheduled before I do any tasks though, ha. So perhaps I would want them to be the first thing so I am reminded to reschedule them... I'll have to think about that more.

I hope this helps!