Can no longer sync and `task diag` throwing tons of "nonexistent task" errors

[sdondley]

To report a bug...

• What command(s) did you run? Ran task sync.

Got an error: Sync failed. The Taskserver returned error: 500 Client sync key not found.

"Hmm, that's weird. Worked fine a couple days ago."

Then ran: task sync init

Got: Sync failed. The Taskserver returned error: 500 ERROR: Could not find common ancestor for abed5db6-500b-49e9-9a4b-1edc10f63165

Ran task sync again: task sync

This time, got:

Sync failed. The Taskserver returned error: 500 ERROR: '253402225200' is not a valid date in the 'm/d/Y' format.

"WTF"?

Ran task sync init again hoping for a different outcome but got:

Sync failed. The Taskserver returned error: 500 ERROR: '253402225200' is not a valid date in the 'm/d/Y' format. again.

Couple more attempts (why does trying the same thing never work) but no dice.

Run task diagnostics

task 2.6.1
   Platform: Darwin

Compiler
    Version: Apple LLVM 13.0.0 (clang-1300.0.29.3)
       Caps: +stdc +stdc_hosted +LP64 +c8 +i32 +l64 +vp64 +time_t64
 Compliance: C++17

Build Features
      CMake: 3.21.3
    libuuid: libuuid + uuid_unparse_lower
  libgnutls: 3.6.16
 Build type: Release

Configuration
       File: /Users/me/.taskrc (found), 3925 bytes, mode 100644
       Data: /Users/me/.task_work (found), dir, mode 40755
    Locking: Enabled
         GC: Enabled
     Server: tw.example.com:53589
         CA: -
Certificate: /Users/me/.task_work/cert.cert.pem, readable, 4042 bytes
        Key: /Users/me/.task_work/key.key.pem, readable, 24742 bytes
      Trust: strict
    Ciphers: NORMAL
      Creds: Blah/Blah/************************************

Hooks
     System: Enabled
   Location: /Users/me/.task_work/hooks
     Active: on-add-pirate    (executable)
             on-modify-pirate (executable)
   Inactive: .DS_Store        (not executable)unrecognized hook name

Tests
   Terminal: 203x42
       Dups: Scanned 1830 tasks for duplicate UUIDs:
             No duplicates found
 Broken ref: Scanned 1830 tasks for broken references:
             Task c3ae8515-4916-4593-ad2b-bc813ac27c20 depends on nonexistent task: [\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"b723baf3-2f3c-42ad-bad9-c9526efa0ecf\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]
             Task b5c467be-e7ff-4239-87c6-4b503bb522e2 depends on nonexistent task: [\"abe46ee9-cdec-4dd0-9093-6783164a9250\"]
             Task 9089c55b-3708-43dd-8735-ae629e885144 depends on nonexistent task: [\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"b723baf3-2f3c-42ad-bad9-c9526efa0ecf\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]
             Task f3c5420c-7b64-43e1-8242-b36d206653b7 depends on nonexistent task: [\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"b723baf3-2f3c-42ad-bad9-c9526efa0ecf\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]
             Task 4b780a7a-c819-4a8b-9b1b-732bd2faa1ad depends on nonexistent task: [\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"[\"b723baf3-2f3c-42ad-bad9-c9526efa0ecf\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]\"]

This goes on for hundreds or thousands of lines.

Tasks are listing fine and everything else appears normal.

[sdondley]

I backed up the tx.data file on the server and then moved it and did a task sync init and everything went smooth.

Looking at the time stamp of the tx.data file, it seems to have stopped updating about 36 hours ago. No idea what I was doing or what could have triggered the problem.

Also of note is the new tx.data file is significantly smaller the old file: 544K vs. 1.8M. Not sure if that's significant or not.

I still have the nonesistent task warnings, though. Looking at them, I'm wondering if those are getting generated by vimwiki which I also use in addition to the command line. Are those anything I need to worry about? Is there a way to get rid of them?

[tbabej]

@sdondley I believe this problem is already fixed on 2.6.2 (which we just released). Consider upgrading and let us know if the issue persits :slightly_smiling_face:

[sdondley]

OK, nice. But which problem are you referring to? All the escaped ids (presumably generated by taskwiki) or my sudden sync failure?

[bradyt]

@sdondley, to get Client sync key not found, I suspect something changed at the tx.data file on the server. I'm guessing either you changed taskd.credentials, or modified the $TASKDDATA/orgs/${credentials.org}/users/${credentials.key}/tx.data file somehow. I'm not sure what official advice is in that scenario. Compare https://github.com/GothenburgBitFactory/taskwarrior/discussions/2740. This issue, https://github.com/GothenburgBitFactory/taskwarrior/issues/1663, seems to suggest that the goal is to work on a replacement, taskchampion-sync-server, rather than consider this an open issue. There is also https://github.com/GothenburgBitFactory/taskwarrior/issues/1937.

Regarding your decision to fix with task sync init, and the resulting Sync failed. The Taskserver returned error: 500 ERROR: Could not find common ancestor for abed5db6-500b-49e9-9a4b-1edc10f63165, consider this discussion that might suggest that task sync init might not be a reliable fix: https://github.com/GothenburgBitFactory/taskwarrior/discussions/2739.

On another project, I've recommended that users not necessarily use task sync init in this way, as Taskserver doesn't seem to support this usecase. You could sync one client to a new clean server account, using task sync init once, then back up each of the other clients, empty the .task/*.data at each one, run task sync to get a working client, then for each of the backed up data, export the tasks sorted by modified date, and try to determine which updates are missing from the new synced clients.

[bradyt]

The escaped ids might be the issue of round-trip of depends at taskd 1.1.0, which I think is fixed at 2.6.2:

- TW #2689 Corruption of the depends attribute upon syncing with taskd 1.1.0
           Thanks to Klaus Ethgen for reporting, Dusting J. Mitchell for
         contributing.

• https://github.com/GothenburgBitFactory/taskwarrior/blob/v2.6.2/ChangeLog#L14-L16

That was an issue where "depends":["a","b"] became "depends":"[\"a\",\"b\"]". That may explain what you're seeing in the escaped ids.

[sdondley]

Thanks for the additional links and details.

Fortunately, I'm the only active client so I should be fine with the task sync init (unless I'm misunderstanding what you mean).

That's good to hear on the escaped ids. I was worried taskwiki might be doing some nasty things to my tasks.

[sdondley]

By the way, I have been doing quite a bit of rejiggering of my taskwarrior settings. I now vaguely remember accidentally commenting out one of the client/server config settings. But I think I caught the problem right away. I can't remember for sure now, though. It's all a blur. I can't even remember if that was today or yesterday.

[bradyt]

Is 253402225200 representing January 11th, 1978? I'm not sure ~Taskwarrior~ Taskserver supports dates before 1980.

[sdondley]

Well, it's a 12 digit number, not 10. So it's probably like several thousands of years into the future.

[sdondley]

Yeah, over 8 thousands years. I did the math.

I'm a good procrastinator, but I'm not that fucking good.

[bradyt]

It sounds like there could be several things wrong with remote and local data. If you have control over the server, you might want to copy tx.data to local, as a backup, and investigate that file for issues. You might also want to back up the ~/.task/*.data files.

EDIT: I overlooked your second comment, you secured tx.data, excellent!

It seems possible that even if you remove the old ~/.task/*.data files, that task sync might result in an error. If the data at server is corrupted, that might be trickier to fix.

Fortunately, I'm the only active client so I should be fine with the task sync init (unless I'm misunderstanding what you mean).

In that case, it sounds like you ran task sync init when the same tasks were already on the server. I suspect this is not officially supported by Taskserver, and minimal reproduce at https://github.com/GothenburgBitFactory/taskwarrior/discussions/2739 might still be related.

[bradyt]

Oh, I just inserted 253402225200 manually into a field in ~/.task/pending.data, and checked task export, it looks like it's 9999-12-31, for example due:someday, and so this should be the bug at https://github.com/GothenburgBitFactory/taskwarrior/issues/2698.

[bradyt]

I finally noticed your second comment. Glad that you seem to have the half of the story where you know how to fix this. One comment in reply:

Why is tx.data smaller now? I think because it no longer has the full history of sync updates. The client sent the current state, and that's all that Taskserver has now. That seems fine.

[bradyt]

You might want to verify that your new Taskserver state can send the same items it just received. You could try moving all your ~/.task/*.data files out of place, and running task sync, Taskserver should try to send all the tasks, and might still error on the 9999-12-31 date.

[tbabej]

The due:someday bug currently requires 1.2 (dev) branch on the server to resolve. That is the next, ehm, task, on my todo list.

[tbabej]

@sdondley In my original message I was indeed referring to the escaping of the depends attribute, as @bradyt points out.

[sdondley]

@bradyt ok, thanks, will have to do that tomorrow or the following day. It's ridiculously late and I'm likely to make a really dumb typing mistake. I almost just accidentally changed the project name on all tasks. Fortunately CTRL-C caught it in time.