My whole Ticktick got messed up on first sync

[zekeadam]

Old completed tasks marked as uncomplete, task names messed up, dates removed, tasks duplicated, my whole ticktick is a mess now. Is there any way to reverse everything the plugin did without going trough everything manually which would take a day?

Edit: Found the backup file, Had to delete and restore everything which got back most of the stuff but all of the time trackings are now unlinked from my projects which sucks.

[thesamim]

Sorry you went through that.

Please updated to 1.0.16. Please turn on debug mode in settings. If this happens again, please send me your console log.

[akseron]

I have the same error, which I suspect is due to timezones and the Tasks note you made about queries vs due dates. My console log contains some sensitive information, but these are tidbits which may help:

{ "id": "65cbe8998f08e37445a1fb5c", "projectId": "65cbe8998f08e37445a1fafa", "sortOrder": -1099511627776, "title": "Verjaardag TickTick/Verjaardagen.md", "content": "", "startDate": "2024-02-20T23:00:00.000+0000", "dueDate": "2024-02-20T23:00:00.000+0000", "timeZone": "Europe/Amsterdam", "isFloating": false, "isAllDay": true, "reminders": [], "repeatFlag": "RRULE:FREQ=YEARLY;INTERVAL=1;BYMONTH=2;BYMONTHDAY=21", "completedTime": "2024-02-08T15:32:20.000+0000", "priority": 0, "status": 0, "items": [], "modifiedTime": "2024-02-13T22:09:29.854+0000", "etag": "pli3cn9x", "deleted": 0, "createdTime": "2024-02-08T15:24:22.000+0000", "creator": 123073621, "tags": [ "verjaardag", "ticktick" ], "columnId": "65cbe8998f08e37445a1fafb", "kind": "TEXT" }

plugin:tickticksync:68642 An error occurred while synchronizing: TypeError: Cannot read properties of null (reading 'path') at FileOperation.addProjectTasksToFile (plugin:tickticksync:67547:57) at async FileOperation.addTasksToFile (plugin:tickticksync:67496:20) at async SyncMan.syncTickTickToObsidian (plugin:tickticksync:68573:22) at async TickTickSync.scheduledSynchronization (plugin:tickticksync:69131:24) at async eval (plugin:tickticksync:59069:11)

The timestamps are NaN minutes apart. Title modified for task 65cbe3288f08e37445a18ee8 "New:" Verjaardag TickTick/D83CDF8AVerjaardagen.md "Cached:" Verjaardag TickTick/D83CDF8AVerjaardagen.md Due date modified for task 65cbe3288f08e37445a18ee8 2024-03-24T08:00:00.000+-100 old: 2024-03-24T23:00:58.000+0000 Task Changed: 65cbe3288f08e37445a18ee8 Task 65cbe3288f08e37445a18ee8 is updated. Content was changed. Due date was changed.

What I found interesting was that the start/due date were detected as February 20th, while it was in fact noted for February 21st on TickTick. My timezone is Amsterdam (GMT+1). I have altered the names of the actual file for privacy reasons.

[thesamim]

@akseron : I've tried reproducing this, but am not able to.

I have: 😁 This folder/💼 Start in emoji folder.md (I've tried with just emoji in folder, just emoji in file name, and emoji in both) And the due dates do not shift. Is it possible you have some non-latin letters in your folder names?

Did find one bug though: updating a due date from Obsidian seems to mess up due date on TickTick. But I don't think that's what you're experiencing. If I'm understanding correctly: you are only dealing with due dates/times set on TickTick. Is that correct?

Opened Issue 62 to track.

plugin:tickticksync:68642 An error occurred while synchronizing: TypeError: Cannot read properties of null (reading 'path')

Right before this line, you should have seen: Creating new file: ${folder.path}/${taskFile} where folder.path is the folder the file would have been created in, and taskFile would have been the file. Could you please let me know if you did, and if you saw anything strange in the way the folder name/file name were displayed?

The timestamps are NaN minutes apart.

This worries me. There should not be a situation where the due date difference results in an invalid number. Opened Issue 63 to track.

Title modified for task 65cbe3288f08e37445a18ee8 "New:" Verjaardag TickTick/D83CDF8AVerjaardagen.md "Cached:" Verjaardag TickTick/D83CDF8AVerjaardagen.md

Q: did you use non-latin characters in the Title? Those two titles look to be the same. However, they did not compare the same....

Due date modified for task 65cbe3288f08e37445a18ee8 2024-03-24T08:00:00.000+-100 old: 2024-03-24T23:00:58.000+0000

In this example, the dates match (March 24th) but there's a 14 hour time difference.

Did you have a specific time in the due date on Ticktick? Was it the AM time or the PM time?

What I found interesting was that the start/due date were detected as February 20th, while it was in fact noted for February 21st on TickTick. My timezone is Amsterdam (GMT+1). I have altered the names of the actual file for privacy reasons.

Could you have been testing close to midnight?

Favor please: Take a back up from TickTickSync settings ("Backup TickTick data"). Take a look at the resulting CSV file.

Take a look at the Due Date, Timezone, Is All Day columns. Do they match your expectations?

[akseron]

Hello Samim, thank you for your reply. I'll gradually answer your questions by updating this comment.

I have: 😁 This folder/💼 Start in emoji folder.md (I've tried with just emoji in folder, just emoji in file name, and emoji in both) And the due dates do not shift. Is it possible you have some non-latin letters in your folder names?

I was quick to jump the gun and have since deleted that comment as the problem had not been solved by this solution.

Did find one bug though: updating a due date from Obsidian seems to mess up due date on TickTick. But I don't think that's what you're experiencing. If I'm understanding correctly: you are only dealing with due dates/times set on TickTick. Is that correct?

Upon the initial sync from Obsidian using TickTickSync, I find that Obsidian first creates files where it goes through every list one by one. Upon looking at the list I see that immediately while Obsidian is still creating its new files, that the dates for (at the very least, maybe others also) full day tasks are one day early. Afterwards when Obsidian is done creating all files in its folder, it reconnects to Ticktick and adjusts the dates in Ticktick according to its own values, afterwards adding the time to the task name together with a calendar emote.

Right before this line, you should have seen: Creating new file: ${folder.path}/${taskFile} where folder.path is the folder the file would have been created in, and taskFile would have been the file. Could you please let me know if you did, and if you saw anything strange in the way the folder name/file name were displayed?

I would say the calendar emoji's along with time, being put in the name of Ticktick tasks after Obsidian is done importing Ticktick tasks. I will try again quickly and comment when what happens.

List of task dates before syncing:

Task created in Obsidian:

Different task while Obsidian is still busy creating tasks:

Task name in Obsidian:

Message in terminal indicating NAN:

Task name in TickTick after Obsidian is done syncing:

List of task dates after syncing:

Pecularities: Now I have looked a bit more in-depth into the errors, full date dates end up shifting in day, however they are not assigned a day or anything. Tasks with either a time are always set to either 13:00 or 14:00 (no idea what determines which one is chosen, 14:00 seems to be chosen more often).

Tasks which have a duration or time set at 0:00, are set to the previous day at 13:00 or 14:00, whereas times that are between 8:00 and 20:00 are set to the same day at 13:00 or 14:00 (maybe a check is used which detects whether the previous day or next day at 13:00/14:00 is closer?).

Tasks without specific dates, but only repeats have no calendar emote with date, added to their names.

These are my time settings:

Now onto the next part which you mentioned regarding csv dates, I think you are onto the right thing, I will divide this into different parts as each type of task seems to have a different error cause, most of them probably stemming back to my date settings:

REPEAT YEARLY:

,"Verjaardagen","Removed name","TEXT","verjaardag","","N","2024-02-20T23:00:00+0000","2024-02-20T23:00:00+0000","","FREQ=YEARLY

Is written in the first slot of the CSV file of the Ticktick backup (before any sync whatsoever). The date listed here is simply wrong and should not be used, I have a proposed solution as looking further in the csv file, the actual correct dates ARE mentioned.

In the fourth slot:

BYMONTHDAY=21","0","0","2024-02-08T15:24:22+0000","2024-02-08T15:32:20+0000","-1099511627776","Europe/Amsterdam","true","false","Not Sectioned","-1","list","42",""

This is probably used by Ticktick to actually import its dates back into its own system. This would mean that either one of my date settings is incorrectly exported by Ticktick, or they are exported incorrectly for all of them (however if this were the case you probably would have experienced similiar errors).

FULL DAY REPEAT MONTHLY WITH ONE COMPLETED: First month, only one column: ,"Uitgaven","Removed name","TEXT","abonnement","","N","2024-02-13T23:00:00+0000","2024-02-13T23:00:00+0000","P0DT9H0M0S","","0","2","2024-02-08T16:43:50+0000","2024-02-14T15:19:55+0000","-1099511627776","Europe/Amsterdam","true","false","Periodieke betalingen","-1","kanban","29",""

Second month, separated into two columns in csv: ,"Uitgaven","Removed name","TEXT","abonnement","","N","2024-03-13T23:00:00+0000","2024-03-13T23:00:00+0000","P0DT9H0M0S","FREQ=MONTHLY

INTERVAL=1","0","0","2024-02-08T16:43:50+0000","","-1099511627776","Europe/Amsterdam","true","false","Periodieke betalingen","-1","kanban","30",""

The correct date is 2024-02-14, which is only stored in the first month, the interval is only stored in the most recent month (first day of the month).

DURATION: ,"LIST REMOVED","REMOVED NAME","TEXT","TAG REMOVED","DESCRIPTION REMOVED","N","2024-02-22T09:30:22+0000","2024-02-22T10:30:22+0000","","","0","0","2024-02-08T21:13:33+0000","","262144","Europe/Berlin","false","false","Afspraken","-1","kanban","7",""

This one contains the correct information in terms of times and dates, which also overlap properly with its timezone. However in Obsidian it is again not stored as the The timestamps are NaN minutes apart. message popped up, storing it at 13/14:00 (probably because of the timezone). +1/2, meaning 13/14 hours from the earliest timezone. The same will probably hold for any task without repeat.

Update: importing one hour later still results in tasks being assigned either 13:00 or 14:00, so it should not be time dependant of of when you are doing it. The titles are also all in latin.

Changing from 24 hour format to AM/PM, also does not help, neither does setting on timezone or week numbers. Maybe a timezone issue which causes it to work for some and for some not?

[thesamim]

Thank you so much for doing all this research. I will got through it in detail shortly.

But first:

Maybe a timezone issue which causes it to work for some and for some not?

I am now convinced it is a timezone issue. Question: is your Obsidian timezone the same as the TickTick timezone?

[akseron]

Hi Samim thank you for your response,

I am not sure, I could not find a timezone setting for Obsidian, but looking around I figured it would probably be the same as my system, in which case yes it should be.

These are my system settings:

Oh! I might be onto something, changing my date settings from English (Netherlands) to English (United States) seems to change CSV export from TickTick! I will now try to sync using this new csv and report back.

Edit: the csv is not changed, the way it is displayed by Excel in Windows is changed! Instead of all the information being displayed in a single tab, it is now allocated into different slots. I will finally try syncing it, and I am fairly confident this will solve everything.

Update: In the isDueDateChangedfunction, there are two tasks imported which are called lineTaskand TickTickTask, TickTickTaskcontains correct information and works as it should, however there is a problem with the lineTask.

dueDate: "2024-02-22T08:00:00.000+-100"
id: "65d091a28f08b8edcbbf5537"
items: []
modifiedTime: "2024-02-17T12:01:00.000+-100"
parentId: ""
priority: 0
projectId: "65d091a28f08b8edcbbf5494"
startDate: "2024-02-22T08:00:00.000+-100"
status: 0
tags: ['ticktick']
timeZone: "Europe/Amsterdam"
title: "removed name     📅 2024-02-22 [TickTick/Afspraken, vaste plannen.md](obsidian://open?vault=Obsidian&file=TickTick/Afspraken,%20vaste%20plannen.md)"
[[Prototype]]: Object

columnId: "65d091a28f08b8edcbbf5495"
content: ""
createdTime: "2024-02-14T12:02:51.000+0000"
creator: 123073621
deleted: 0
dueDate: "2024-02-22T13:20:00.000+0000"
etag: "kuekcd3e"
id: "65d091a28f08b8edcbbf5537"
isAllDay: false
isFloating: false
items: []
kind: "TEXT"
modifiedTime: "2024-02-17T10:59:46.954+0000"
priority: 0
projectId: "65d091a28f08b8edcbbf5494"
reminder: "TRIGGER:PT0S"
reminders: [{…}]
repeatFlag: ""
sortOrder: -1099511693312
startDate: "2024-02-22T13:20:00.000+0000"
status: 0
tags: ['ticktick']
timeZone: "Europe/Amsterdam"
title: "removed name [TickTick/Afspraken, vaste plannen.md](obsidian://open?vault=Obsidian&file=TickTick/Afspraken,%20vaste%20plannen.md)"
[[Prototype]]: Object

When creating a date1 and date2 variable with the both of them, these are the dates which are used for lineTaskDueand TickTickTaskDuerespectively: 2024-02-22T08:00:00.000+-100, 2024-02-22T13:20:00.000+0000

date1is assigned Invalid Date date2is assigned Thu Feb 22 2024 14:20:00 GMT+0100 (Central European Standard Time)

Which in turn results in the NAN minutes apart, as date1 does not have a correct date set. This might have to do with the initial creation in Obsidian being wrong, and afterwards any references to it also being incorrectly saved as a result.

Which might have to do something with this error:

plugin:tickticksync:68642 An error occurred while synchronizing: TypeError: Cannot read properties of null (reading 'path')
    at FileOperation.addProjectTasksToFile (plugin:tickticksync:67547:57)
    at async FileOperation.addTasksToFile (plugin:tickticksync:67496:20)
    at async SyncMan.syncTickTickToObsidian (plugin:tickticksync:68573:22)
    at async TickTickSync.scheduledSynchronization (plugin:tickticksync:69131:24)
    at async eval (plugin:tickticksync:59069:11)

Edit: Alright, while trying to solve the errors, I have randomly had one sync go through just fine, so this is quite confusing. The only thing I could note was different, was that I had forgotten to delete the separate Inbox folder which was imported as a list instead of the one made by Ticktick.

[akseron]

Tickticksync error message when creating tasks while only the inherent Ticktick inbox exists:

Tickticksync error message when creating tasks with a separate Inbox list:

The error might just stem from the Default project setting:

Well in any case the presence of a separate Inbox list seems? to fix the bug.

Trying an entirely differently named folder:

Error still occurs: presence of a file named Inbox.md in your lists is essential.

Edit: Tested around, Ticktick syncs for me properly as long as both Inbox lists are the exact same, the moment you add one item to one of the two, the entire syncing messes up and The timestamps are NaN minutes apart. start flooding.

[thesamim]

@akseron : I'm not sure I've taken care of all the issues. I can only do limited testing with timezones...

Favor please. The attached version of the plugin has way more logging to help me understand what's going on.

Could you please:

1. Change your settings so that you are only synchronization one project (so that if it's not right, we're limiting the damage to one project.)
2. Make a copy of your current main.js file in <vault directory>\.obsidian\plugins\tickticksync
3. extract the files in the attached zip file to that directory
4. make sure that debug mode is on
5. test again
6. in the icon bar, you will see a new icon . When you click it, you will see a popup notice with your time zone.

Please let me know what happens with the testing, and what your time zone is set to in the popup notice.

Thanks in advance.

tickticksync.zip

[akseron]

The two times both have the same timezone now, however their timestamps are 1 day apart, meaning all my dates are placed one day earlier. There is some improvement however, as they do retain the correct time at 0:00 instead of being put to 13/14:00.

These are error messages which occurred:

All days are imported one day early:

Error messages on a second run where I changed a setting in main.js from America/New_York to Europe/Amsterdam (probably did nothing):

Edit: the icon appeared

Somehow the synced list ended up disappearing after pressing login again and prompted this window:

[thesamim]

@akseron One more time please. There was a bug with converting UTC to Local time. I hope I fixed it.

I could not reproduce:

These are error messages which occurred:

If you can reproduce it: Please let me know the exact sequence of events that go to that error.

Could not reproduce the following either. All those errors indicate that the TickTick servers were having issues at the time.

Error messages on a second run where I changed a setting in main.js from America/New_York to Europe/Amsterdam (probably did nothing):

Again: if you can give me reproduction steps, I would be glad to fix (or mitigate if I cannot fix.)

Edit: the icon appeared

Good! No surprise there.

Somehow the synced list ended up disappearing after pressing login again and prompted this window:

There is no code in TickTickSync that would trigger that screen. Which leads me to a question I should have asked from the beginning: What other plugins are you running?

Regardless: Please try again with the attached. tickticksync.zip

[akseron]

I could not reproduce:

These are error messages which occurred: image

If you can reproduce it: Please let me know the exact sequence of events that go to that error.

These errors usually occur after I reinstall Tickticksync. I usually attempted to entirely delete both Tasks and Tickticksync every time to ensure that any old tasks were removed from the system. After some experimenting I was contemplating that this caused Tickticksync to not be properly setup, which would also explain the fact that the icon was not appearing at first. However, even after Tickticksync was setup as it should be (as far as how a layman would set it up), the syncing still went wrong at some point (If this happens again I'll document an example).

Could not reproduce the following either. All those errors indicate that the TickTick servers were having issues at the time.

I am guessing that Tickticksync really needs a reboot or two before being setup entirely considering this is what I usually needed during experimenting for the icon of before to appear (I will mention whether this is still the case in a bit).

There is no code in TickTickSync that would trigger that screen. Which leads me to a question I should have asked from the beginning: What other plugins are you running?

I am using the following list of plugins:

At an earlier stage of experimenting, I did try turning off all plugins and I believe (not too sure however) that the bug still happened. It sometimes occurred when I tried pressing Login / setting a folder by using the + button. I will be turning trying your solution with all plugins turned on, and in case it does not work I will try again with all plugins turned off.

Regardless: Please try again with the attached. tickticksync.zip

Will do!

[akseron]

1. Installed TickTickSync and did not touch any setting yet.
2. Closed Obsidian and changed the plugin files with yours. (Getting a prompt 'Please login from Settings.', cannot see your icon yet)
3. Pressed login once, already logged in in the pop-up screen, however TickTickSync still displays login setting.
4. Pressed login again, Tickticksync has not detected a login (repeat this about 10 times)
5. Trying to log out from the already logged in pop-up.
6. Pressing login after logging in from the pop-up works now and Tickticksync is loaded succesfully.
7. Set Default folder location to Ticktick/Tasks
8. Set Project to Verjaardagen, receive a notification Did not find a default Project File for Project Verjaardagen. Please create a file and set it's default to this project, or select a file to be the default for this project. 9. Set Default project to Verjaardagen
9. Turn on Debug mode
10. Check database
11. At this point I notice that your icon still has not appeared, so I will restart Obsidian.
12. Upon restarting Obsidian, the icon is loaded, an icon called TickTickSync is also loaded.
13. Sync using TickTickSync.

Beginning of the developer console:

Ending of the developer console:

Looking at the dates I see that each and every one of them is set to the correct day, for some reason the ones which are set to 00:00 are changed to 12:00, which I am not sure of the reason behind. The full day tasks are correct.

15. While typing my comments above, Tickticksync syncs from Obsidian back to Ticktick and updates every date to be the same as the one created in Obsidian.

Example of one change:

16. Trying to refresh in Ticktick created a new file with sync-conflict at the end, one date was set to 24:00 in the original Obsidian file: and set to 12:00 one day later in the sync conflict file:

The original date in Ticktick was: 2024-11-22 at 00:00 (not full day), its time in the Ticktick backup file is set at 2024-11-21T23:00:30+0000, Europe/Amsterdam. The only noteworthy thing about it, is that it is the last date which is not a full day date.

This is how the task is synced in Obsidian:

17. I'll move on and try to sync a different list, to see how that responds. However I imported the incorrect file and now I have some duplicate lists (noticed that at step 21)
18. I change the Default Project Folder to my other list.
19. I change the Project to my other list.
20. I try pressing Login to ensure that Ticktick is still connected
21. The error from last time happens, random files are opened, There are duplicate files and after sometime a message is shown that Tickticksync failed to login.
22. The errors stopped and Ticktick is now logged in.
23. I specify the Default folder location to ensure everything is setup properly.
24. I press login again, and the error occurs again (will not be pressing it anymore). I delete all duplicate lists and restore the original Verjaardagen file.
25. I press sync, the sync completes and I am prompted to delete the old tasks from my first attempt.
26. For this part I'll make a quick summary, the tasks synced properly without a conflict this time. Tasks with a specific time synced fine. Tasks with a duration however, discarded the duration and ended up using the end date as their times. The days were all correct.

[thesamim]

So... Those dates now look correct to me. Are they not what you're expecting?

regarding the number of times it took to get it to recognize login:

1. What OS are you on?
2. Is the file system local, or are you using a cloud service (like onedrive or iCloud)
3. Do you have any software running synchronization in the background with another filesystem?

[akseron]

1. Windows 11
2. I have it synced up through Syncthing
3. I have my phone (Android) connected through Syncthing

[thesamim]

@akseron : That's pretty much my setup, except I have my vault on a onedrive folder. I have not encountered those issues. Please check your syncthing log to see if there are any conflicts there.

About the dates: are they matching up with what's on TickTick now?

[akseron]

@akseron : That's pretty much my setup, except I have my vault on a onedrive folder. I have not encountered those issues. Please check your syncthing log to see if there are any conflicts there.

I will try to do so when I have some spare time.

About the dates: are they matching up with what's on TickTick now?

Yes and no, the dates end up being synced correctly, however the times associated with the dates are incorrect.

[thesamim]

@akseron

Yes and no, the dates end up being synced correctly, however the times associated with the dates are incorrect.

One more time please. Do not be alarmed if your tasks get updated on first run. It'll fix up any cached tasks that way...

I think I found the time issue.

But I'm still not sure why updating the main.js is causing so many issues.

tickticksync.zip

[akseron]

But I'm still not sure why updating the main.js is causing so many issues.

I figured out how the error occurred with regards to login. I used the following way to recreate the error: use Tickticksync properly without any sync yet, login and create a folder to save your files by clicking on the + button. Afterwards restart Obsidian, and before Tickticksync is done synchronizing, delete the folder you previously created. After Tickticksync synchronizes you repeat the steps to create the same folder by clicking on + and afterwards press login and voila, the error occurs. This is not dependent off any other plugins, however plugins are triggered when the error happens. The reason why I had it occur to me, is that oftentimes Tickticksync took a longer time for me to sync somehow, however in a regular case the chance of someone deleting the file in this short interval before Tickticksync is done synchronizing, is fairly low I believe.

Untitledand Untitled canvas are created in case you delete the Tasks file here, before Tickticksync is done synchronizing:

These were the settings in Tickticksync:

There are no errors while the Untitled files are created, however if you were to delete them and afterwards check the database this message pops up: Perhaps suggesting that these were created as tasks?

So that should clear things up a bit.

[thesamim]

Cool. Thanks. Will keep an eye on that.

Now: How are the dates and times looking on the tasks?

[akseron]

Okay took me a bit to get everything running, but I changed my private values inside the files so that I could share some direct data copies with you.

I ran the sync 5 times, without touching anything in the Obsidian or Ticktick dates. I kept seeing something change, so I kept running it until I realised one date kept having a day added on top, thus stopping my runs as I figured it was not necessary to continue.

The problem remains that dates are set correctly, however the time is set at 12:00 instead of 00:00. Also the last task in the list seems to keep getting a date added everytime a sync is completed.

Here are the dates and csv data before: TickTick-backup-2024-02-21-special-test-before.csv

Here are the dates and csv data after: TickTick-backup-2024-02-21-special-test-after.csv

Here is the log: obsidian.md-1708476197899.log

Edit: almost forgot, also a screenshot of the Obsidian file.

Edit2: running it again and taking a screenshot of the obsidian file after every sync: sync 1:

sync 2:

Upon testing it again it seems to work properly in Obsidian, something probably went wrong while downloading the new file. In Ticktick the times are properly noted!

[akseron]

Checked for another file which only has durations inside of it, all the correct dates were picked, however the durations in Ticktick and Obsidian were both changed to only the start time, with no end time.

[thesamim]

@akseron : please try 1.0.17 update Thanks for All your help. Also please check out the Read me file acknowledgment section. 😁