may be only sync the undo task?

[Tback1]

shall we add a selection only sync the undo task.

the sync download too much task to one file.and it add the #ticktick and badly it upload again ,even more worse the Ticktick Automatically identifying strings starting with # in the tasks and re uploading directly caused a tag disaster

Make the task list chaotic and the downloaded file contains too many entries, causing the rendering process to freeze the observer directly

this image is I sync by the 1.0.13 wait for 10min downloaded all yesterday.

Synchronizing too many entries at the same time is prone to denial of service on the one hand, and initialization failures on the other hand

finally I sync in 1.0.14, but I will not

[thesamim]

finally I sync in 1.0.14, but I will not

I'm sorry: you will not what?

The problem with only syncing open tasks, with the current architecture: keeping track of which tasks are actually downloaded so that if you close them in TickTick they will also close in Obsidian.

The other problem is that TickTick does not provide a way to only get open tasks, so there will still be a network delay fetching everything. But I think I'm hearing you say that the bigger problem is all the processing that happens after the tasks are download and the update of the files taking so much time...

If it's a big enough issue, I could:

1. Make downloading only currently open tasks a setting
2. Track which tasks we are actually downloading
3. Only look for Tasks that are download to do status (open/close) updates from TickTick.
4. Closing Tasks from Obsidian would not change.

Let me know what you think.

I'm concerned about the number of "Parent ID not found in file" errors. Could you please check on TickTick if somehow you ended up with duplicate lists in Ticktick?

[Tback1]

It's mix too much question to say. I will not is nothing ,just forget to delete in the edit. in the first the sync not complete too long,nut suddenly it start output the {project name}.md, so the edit is hurry and confusion.I'm sorry that make you misunderstand.

Back to the question My inbox had 651 entries and full is 774. maybe it's too hard for the deal for sync, by the way almost 50 is add by the 1.0.13, it's duplicated some entries apparently。

I agree the opinion the tasks are download and the update of the files taking so much time, because the backup.csv had the full content of database, the download is working, but when I use VSCode to find the [link][https://www.dida365.com/xxxx] and #ticktick in the {project name}.md had found 1053 item. So it's duplicated almost 300 entries, maybe a reason is I delete the #ticktick in the dida's web before, but I use a new folder 滴答清单 start the new sync, not the old one, but it's make the follow error too.

now my ticktick database is full of error entries. so many entries become this

So, after this adventure My Advice :

1. so still now we don't have a nature way to just download processing, so could we add a setting just download first ,not to sync initial
2. only sync the new task after download and the old task add #ticktick manually.
3. if the user need in the obsidian to manage the task, after the sure and setting just give them the double sync. In my opinion, I could just to use obsidian to backup and clean the ticktick entries. just need information flow from ticktick to obsidian

[thesamim]

Regarding duplication: That's a big issue. I've seen it before, but I thought it was because of the strange way I was testing. Still investigating why it happens. Could you please check on the dida web if you now have a number of duplicated lists?

Regarding managing the volume of updates: I had an idea on how to manage it:

1. In settings: Add setting to only download open tasks
2. In settings: Add a limiting number. (eg: only sync 300 tasks at a time)
3. Processing: if a task is marked complete in dida/ticktick, and it is in Obsidian, then mark it as complete in Obsidian IF it has been synced previously, otherwise ignore the update.

The thinking is: When syncing only open tasks, the additional logic needed will be to NOT add completed tasks to Obsidian if the setting is set, and only update tasks already synced. Limiting the number of tasks to update per Sync operation will address the concern about taking too much time. As designed, the sync will pick up further updates on the next sync cycle. Eg: if the limit is set to 50, and there are are 300 open tasks in dida/ticktick, it will take 6 sync cycles to complete the operation.

I think that addresses the concern, but it will take some serious testing to make sure I get it right.

[Tback1]

I reviewed my operation and I think there is a reason for this failure, and the process is as follows:

1. I first tried to sync only one of the list projects. At the same time, the project returned the same project list, but as I described earlier, this operation keeps failing.
2. But unfortunately, the prompt is insufficient, when I see the console's Sync Completed message, I decried and changed the sync return to inbox.
3. This synchronization waited for more than ten minutes and was successful, but the successful result was a confused to do list.

---

Regarding managing the volume of updates: I had an idea on how to manage it:

1. In settings: Add setting to only download open tasks
2. In settings: Add a limiting number. (eg: only sync 300 tasks at a time)
3. Processing: if a task is marked complete in dida/ticktick, and it is in Obsidian, then mark it as complete in Obsidian IF it has been synced previously, otherwise ignore the update.

I think this approach may have some effect, but it may be a palliative rather than a cure. Reducing a single entry does not guarantee that the synchronization will be correct

In the part I understood, it seems that the database that is synced down is the entire library, as can be seen from the backup.csv. The problematic part arises while processing the data to convert to an MD file. The problem is that it takes too long to process convert to MD, and the default cycle is only 300. So on the one hand, the problem is caused by the synchronization initiated before the initial download is completed, and on the other hand, the obsidian jump link, tag , and sync id were attached to the task, which further exacerbates the problem. more as I did, changing the sync folder after the console may be a wrong completion message is the cause of a catastrophic accident.

---

so I think the most important is

1. not add tag in the download keep it original.
2. I see the id additional to sync id , which use %% 24bit %% , also put it into content not the title of a task. (by the way Those additions cause the dida's web page to freeze accordingly and render delayed. Even if these appended entries are checked and put into Completed, the situation will not be improved, and the above entries must be deleted to solve the problem)
3. Just let the user sure they download the full of the online content, on that base could decide start the sync.
4. maybe we need a tool to check it, not the check database,it confuse me,pop 1000 more different,( no diff , only ignore or delete, and the long name with id and #ticktick, it's madness ) . the tool will batch to deal the different.
5. after the download just deal with new not change the old too much.(and may some user like me just want to use it as a storage of to do)

I could help the test , but I will use a new database to do it, I hate Data Recovery。┭┮﹏┭┮

[thesamim]

I reviewed my operation and I think there is a reason for this failure, and the process is as follows: [...]

Yup, that's what caused the confusion. And the reason is: current version of TickTickSync did not handle moving tasks between projects, or changing the parent of a task. Version 1.0.15 (the version I'm working on right now) will handle this situation. Stand by.

I think this approach may have some effect, but it may be a palliative rather than a cure. Reducing a single entry does not guarantee that the synchronization will be correct

Let's see what happens with the new version.

In the part I understood, it seems that the database that is synced down is the entire library, as can be seen from the backup.csv.

Just FYI: The backup CSV file is just for back up. It's not used for processing.

The problematic part arises while processing the data to convert to an MD file. The problem is that it takes too long to process convert to MD, and the default cycle is only 300.

It might be worthwhile for you to bump up that up to 900 seconds (15 minutes) to get past that issue.

1. not add tag in the download keep it original.

2. I see the id additional to `sync id` , which use `%% 24bit %%` , also put it into content not the title of a task.
   (by the way **Those additions** cause the **dida's web page to freeze accordingly and render delayed**. Even if these appended entries are checked and put into Completed, the situation will not be improved, and the above entries must be deleted to solve the problem)

Those are added in the title, and not the content, because I have had a request to NOT alter the content (items) of the task. Adding them to the content messes with a whole bunch of workflows....

The processing relies on the Tag and IDs being in the task because there's three datasources involved in the synchronization:

1. The .md files
2. The local cache (currently in data.json)
3. The TickTick data (ie: what's in the cloud)

Briefly: TickTickSync compares all three to determine what changes are made where and when and act on those changes. ie:

1. Did the user change a task in .md? Determined by comparing 1. and 2. Update TickTick
2. Did the user change a task in TickTick? Determined by comparing 2. and 3. Update .md

Sometime in the future, I will consider using a real database to have only one source of truth but that's a whole redesign.

4. maybe we need a tool to check it, not the check database,it confuse me,pop 1000 more different,( no diff , only ignore or delete, and the long name with id and #ticktick, it's madness ) . the tool will batch to deal the different.

That was the intent of the "Check Database" button in Settings. Full disclosure: it only works partially. That's next on my agenda...

I could help the test , but I will use a new database to do it, I hate Data Recovery。┭┮﹏┭┮

Understood. I appreciate your patience so far.

Couple of recommendations for now:

1. Please wait until 1.0.15 comes out to retest. If all goes well, that should be later this week.
2. Please change your sync interval to 900 secs

After 1.0.15 comes out we'll evaluate the need for throttling (ie: limit number of tasks to sync per sync interval....)

[thesamim]

@Tback1 : could you please test with1.0.18 and let me know if we still need throttling?

[thesamim]

Not having heard further on this issue, closing until it is reported again.