Evolution+DecSync doesn't show all events, Android+DecSync CC

[java-py-c-cpp-js]

I have 5 calendars for different purposes, one with a lot of events and a large part of it are repeated periodically. They are not stored in the default folder but in ~/DecSync. My current problem is, that the most events (some repeated periodically, some not) are not displayed in evolution even if I right-click on one of the calenders and press refresh. Direct after the import of the calender data, more events where showed, but not all. After I restarted Evolution, only a few remained.

I created the calendars with DecSync CC on Android and sync them with Syncthing. If I sync them to another Android device, it takes some time to load, but then all events are showing up.

The problem occurs after an evolution update there they say they have changed something on the format calendars are stored or something like that (sadly, I didn't remember the whole pop-up message - sorry), so maybe that causes the problem.

Sadly I can't provide my DecSync folder as an example because there is a lot of private stuff in it but maybe I manage to reproduce the error with some test calendar somewhere in the future.

Evolution: Version 3.34.1 Linux-Version: Manjaro (Arch based)

[39aldo39]

Thanks for the detailed report! I understand that you cannot share the DecSync folder, but could you try adding the calendar a second time? Having the same calendar multiple times does work. Maybe it does work then, or maybe there are different events missing.

[java-py-c-cpp-js]

Thanks for your answer! I added the calendar a second and a third time, and exactly the same entries are shown or missing in each copy. Restarting Evolution with closing all related processes (evolution-calenar-factory and so on) with some kind of "taskmanager" doesn't change anything.

[39aldo39]

It would be very useful to get the value of such entries to see whether I can reproduce it. I have created a nice one-line script to get the event data of a specific event. You have to replace 'NAME' by the name of the event, or another unique property:

grep -rl 'NAME' ~/DecSync/calendars/*/stored-entries | head -n1 | xargs jq '.[2]' | xargs echo -en

For this, you need to have jq installed. You probably want to censor some personal information in there with XXX or similar. A large portion will probably be timezone information, denoted between BEGIN:VTIMEZONE and END:VTIMEZONE.

If you encounter any problems doing this, just let me know.

[java-py-c-cpp-js]

I does some spot tests with your one-liner and found out that the not shown entries I tested where created with Evolution (PRODID:-//Ximian//NONSGML Evolution Calendar//EN) in a time it was showing all entries, while the shown entries are created with Android PRODID:+//IDN bitfire.at//ical4android. If I create a new entry with evolution, it is shown (PRODID:-//Ximian//NONSGML Evolution Calendar//EN), even in a complete new installation in a new user account in Manjaro. (By the way: In the complete new user account, the same entries are not shown as in my normal user account)

[java-py-c-cpp-js]

Example 1 - it is not shown:

BEGIN:VCALENDAR
CALSCALE:GREGORIAN
PRODID:-//Ximian//NONSGML Evolution Calendar//EN
VERSION:2.0
BEGIN:VEVENT
UID:56XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
DTSTAMP:20190916TXXXXXXZ
DTSTART;TZID=/freeassociation.sourceforge.net/Europe/Berlin:
 20191002T123000
DTEND;TZID=/freeassociation.sourceforge.net/Europe/Berlin:
 20191002T140000
SEQUENCE:3
SUMMARY:XXXXXXXXXXXXXXXXXXXXXX
LOCATION:XXXXXXXXXXX
TRANSP:OPAQUE
CLASS:PUBLIC
CREATED:201909XXTXXXXXXZ
LAST-MODIFIED:201909XXTXXXXXXZ
RRULE;X-EVOLUTION-ENDDATE=202003XXTXXXXXXZ:FREQ=WEEKLY;COUNT=24;BYDAY=WE
END:VEVENT
END:VCALENDAR

Example 2 - it is shown:

BEGIN:VCALENDAR
CALSCALE:GREGORIAN
PRODID:-//Ximian//NONSGML Evolution Calendar//EN
VERSION:2.0
BEGIN:VEVENT
UID:c2XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
DTSTAMP:20191129TXXXXXXZ
DTSTART;TZID=/freeassociation.sourceforge.net/Europe/Berlin:
 20191127T200000
DTEND;TZID=/freeassociation.sourceforge.net/Europe/Berlin:
 20191127T203000
SEQUENCE:3
SUMMARY:Test with new evolution install 2
TRANSP:OPAQUE
CLASS:PUBLIC
CREATED:20191129TXXXXXXZ
LAST-MODIFIED:20191129TXXXXXXZ
RRULE;X-EVOLUTION-ENDDATE=20200506T180000Z:FREQ=WEEKLY;COUNT=24;BYDAY=WE
END:VEVENT
END:VCALENDAR

Example 3 - it is shown:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:+//IDN bitfire.at//ical4android
BEGIN:VEVENT
DTSTAMP:201911XXTXXXXXXZ
UID:73XXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
SUMMARY:XXXXXXX
DTSTART;VALUE=DATE:201911XX
DTEND;VALUE=DATE:201911XX
STATUS:CONFIRMED
BEGIN:VALARM
TRIGGER:-PT720M
ACTION:DISPLAY
DESCRIPTION:XXXXXXX
END:VALARM
END:VEVENT
END:VCALENDAR

[java-py-c-cpp-js]

The problem occurs after an evolution update there they say they have changed something on the format calendars are stored or something like that (sadly, I didn't remember the whole pop-up message - sorry), so maybe that causes the problem.

In issue #15 there is a description of the changes mentioned by the error message:

It looks like there are indeed some changes in the Evolution Data Server. I will update the backend code to incorporate the changes. Thanks for the notice!

[java-py-c-cpp-js]

Entries that can be read on Android but not with Evolution are only in a~/DecSync/calendars/colID<some_id>/stored-entries/XXXXXX-DecSyncCC/ folder. Entries that can be read with Evolution, too, are also in 3 different ~/DecSync/calendars/colID<some_id>/stored-entries/XXXXXXXXXXXXXXXXX-pc-Evolution-<some_number>/ folders. I think there are different XXXXXXXXXXXXXXXXX-pc-Evolution-<some_number> folders with different numbers because I reinstalled Evolution two times (and deleted all settings data of Evolution while doing that).

[39aldo39]

Entries that can be read on Android but not with Evolution are only in a~/DecSync/calendars/colID<some_id>/stored-entries/XXXXXX-DecSyncCC/ folder.

This is very strange. When you refresh, it should add the entries in a ~/DecSync/calendars/colID<some_id>/stored-entries/XXX-pc-Evolution-<some_number>/ folder. I'm not sure how this can happen. Could you try deleting the folder ~/DecSync/calendars/colID<some_id>/read-bytes? (Removing it for all apps is fine.) Do they at least appear in the stored-entries folder when you refresh?

[java-py-c-cpp-js]

Some entries are now appearing in Evolution because I have changed them on my smartphone. Other still doesn't, even after I run rm -r ~/DecSync/calendars/colID*/read-bytes and refreshed. I first refreshed on Evolution, then on my Android device, then in Evolution again.

Do they at least appear in the stored-entries folder when you refresh?

After all the deleting and refreshing, I tested one of the not appearing entries. It only appears in the ~/DecSync/calendars/colID<some_id>/stored-entries/XXXXXX-DecSyncCC/resources folder, but not in the one for Evolution.

BTW, just because maybe it's important: Entries count on Android:

Android entries: 138
Processed entries: 139
DecSync entries: 139

Is it possible and would it make sense to get the entry count in Evolution to fix this problem?

[39aldo39]

I still have no clue why it doesn't even try to read the entries. It is a bit concerning as it a basic function that should always work. Luckily it seems like you are the only one which such a problem (for now)...

As it only concerns old entries, you might just try exporting and importing the calendar data on your phone using something like Calendar Import-Export, with the import into a new collection. As changes from your phone do work, the entries should appear.