Duplicated tasks with same ID on import

[nick87720z]

I could not find any bugs about duplication problem. Duplication was noted in different, now closed, bug, though not replied.

Version information. Using release 4.7.1. My version is from gentoo, so I ensured, that there are no patches, affecting described bug: https://data.gpo.zugaina.org/gentoo/app-office/calcurse/files/

Bug description. When importing, all tasks are added even if same task already exists after previous import. However, I noticed, both from apts file and from imported ical file, that tasks have IDs. In apts file - duplicated tasks have same ID, looking like sorted by this ID. However, I don't see any options about how to handle duplicates on import.

Reproduce.

• Get ical file for import (may be export existing data).
• Import multiple times.

Expected Behavior. Result: tasks should be not duplicated after above steps. By default (if no option not set or implemented) task or entity with same ID should be replaced. I see that already happens for notes - but it could not happen, because notes are saved in separate files with ID as filenames, which tasks are not. So, replacement woult be best default option.

If option is implemented - there could be choice: replace or not. If could also add interactive question with option to not ask again, with such format, where R letter or similar is added to main anwser (y/n/), meaning "remember" (don't ask again).

Update: I removed screenshot, featuring IDs from apts file, as they are of no use there. I misinterpreted them as task ID, but it seems to be rather notes ID. Tasks ID are not in apts if ever exist.

Update: I checked ical file. There are seem to be indeed some even ID, which don't repeat between tasks. In contranst - I could not find any source for notes IDs. They seem to be combined from DESCRIPTION and LOCATION, while SUMMARY is used for event name in tasks list.

Update: found confirmation, that those UIDs are reliable way for deduplication: https://www.kanzaki.com/docs/ical/uid.html Not sure though, were is source for these specs.

[lhca]

For clarification, I assume that what you call a "task" is named a "To-Do" in the iCalendar specification RFC 5545. An ical file contains one or more iCalendar objects which in turn consist of one or more calendar components. When you import an ical file, calcurse will extract event and To-Do components. There may be other not supported components which are silently skipped. An event component is imported as an appointment. A To-Do component is imported as a todo.

Event and To-Do components have various "properties" (described in the ical file) some of which are used by calcurse while others are ignored. Both event and To-Do components have a property called a UID which according to RFC 5545

defines the persistent, globally unique identifier for the calendar component.

The UID is not imported or used by calcurse neither for events nor for To-Do's. Hence repeated import of the same ical file will create duplicated appointments and todos. Also, calcurse-exported ical files do not contain UIDs in violation of RFC 5545.

This is unlikely to change as it will require a major design change and quite a lot of code. Among other things, the calcurse-caldav application makes use of UID for internal housekeeping purposes.

[bolt-blue]

I am having afaict, exactly the same issue. @lhca So, if we e.g. have a script to retrieve .ics file(s) and then import them, we have no way to prevent entries from being duplicated? If this is the case, it's sadly a bit of a deal breaker.

[bolt-blue]

As nick87720z pointed out, some form of unique ID's are present in apts (and presumably todo). I don't see my handling de-duplication on import would require any major overhaul. Even passing e.g. apts through uniq does the trick, but it feels hacky to do this independent of calcurse.

[cerisara]

I support that this "deduplication" on ical imports would be a nice feature to have! In the mean time, may be someone already developed an external script to take care of that?