strictness of iCalendar imports

[mvhulten]

test.ics.txt calcurse_log.WXqMzB.log

Hello,

I could not import an iCalendar event. I shortened the real event to a more minimal (and less private) example (attached). I typed:

calcurse -i test.ics

and got the error in the attached log file. It says that DTSTART could not be found for VEVENT. When I add a DTSTART line just after BEGIN:VEVENT, it works (the item is imported without error).

I guess DTSTART is needed after any START: line. Is this right?

Would this mean that there is a bug in Apple's iCalendar library (it says Mac OS X 10.15.7 in test.ics)?

When import fails, like in this case, calcurse -i returns an error code of 0, so it's not easy to catch the error in a script.

I am using calcurse 4.7.0 (self compiled on RHEL 7.9).

Thanks,

—Marco

[lhca]

test.ics.txt calcurse_log.WXqMzB.log

Thank you for this report from the real cyber world.

I will give some comments on the test file. Numbers in parenthesis refer to section numbers in RFC 5545, the iCalendar specification.

The file contains an iCalendar object (3.4). An iCalendar object consists of some general properties pertaining to the entire object (3.6):

VERSION:2.0
PRODID:-//Apple Inc.//Mac OS X 10.15.7//EN
CALSCALE:GREGORIAN

and one or more components, here two. A time zone component (3.6.5):

BEGIN:VTIMEZONE
TZID:Europe/Oslo
BEGIN:STANDARD
TZOFFSETFROM:+0200
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
DTSTART:19961027T030000
TZNAME:CET
TZOFFSETTO:+0100
END:STANDARD
END:VTIMEZONE

and an event component (3.6.1):

BEGIN:VEVENT
DTEND;TZID=Europe/Oslo:20201027T120000
DTSTAMP:20201020T134603Z
LOCATION:Home office
DESCRIPTION:A test meeting.
SEQUENCE:0
SUMMARY;LANGUAGE=us-EN:Interview 2nd round
END:VEVENT

There are several other types of components: to-do, journal, free/busy and alarm (3.6), but calcurse only supports events and to-dos. Other components are silently ignored, here the time zone component (which by the way looks fishy).

I guess DTSTART is needed after any START: line. Is this right?

Would this mean that there is a bug in Apple's iCalendar library (it says Mac OS X 10.15.7 in test.ics)?

The event is missing a DTSTART property which in this context is required (3.6.1). It tells when the event (meeting) begins.

When import fails, like in this case, calcurse -i returns an error code of 0, so it's not easy to catch the error in a script.

That should definitely be corrected.

[j95io]

I have the same issue that @mvhulten described. Maybe he messed up his "minimal failing example" here, but the issue persists.

If the line DTSTART:... is not immediately after BEGIN:VEVENT, then calcurse doesn't import the event.

In the MS Exchange I'm supposed to use, they generate ical files where DTSTART: .. lines comes after the SUMMARY:... line and not right after BEGIN:VEVENT.

Try out the ones I'm attaching works.ics.txt worksnot.ics.txt

[lhca]

I have the same issue that @mvhulten described. Maybe he messed up his "minimal failing example" here, but the issue persists.

I will assume that @mvhulten provided a valid test example of a failure. So yours is a different problem.

If the line DTSTART:... is not immediately after BEGIN:VEVENT, then calcurse doesn't import the event.

If DTSTART comes after DTEND, DURATION, EXDATE or RRULE, calcurse will complain as shown in @mvhulten's log file. The iCalandar standard RFC 5545 imposes no ordering on these so-called properties, hence calcurse violates RFC 5545 in this respect since v4.7.0. This will have to be changed.

The pragmatic reason for the requirement is that it makes parsing of an icalendar object easier: the properties mentioned must agree with DTSTART in value type. All examples in RFC 5545 have DTSTART before the others, but that's no excuse.

Thank you for providing the Microsoft Exchange Server example. It is interesting in other respects. The DTSTART property has a time zone identifier parameter:

DTSTART;TZID=W. Europe Standard Time:20201029T110000

The TZID value will not be recognized by calcurse (probably). What was the import result? In particular, the start and end times of the appointment?

[j95io]

If DTSTART comes after DTEND, DURATION, EXDATE or RRULE, calcurse will complain as shown in @mvhulten's log file. The iCalandar standard RFC 5545 imposes no ordering on these so-called properties, hence calcurse violates RFC 5545 in this respect since v4.7.0. This will have to be changed.

... Yeah would be awesome. So in my case, MS Exchange put the RRULE before DTSTART and therefore I couldn't import it to calcurse, I see

The TZID value will not be recognized by calcurse (probably). What was the import result? In particular, the start and end times of the appointment?

Thankfully calcurse ignores the TZID. AFAIK "W. Europe Standard Time" is what Britain and Portugal have, However the time zone info like TZOFFSET shows the central European time zone.

The central European one is also what is used when importing, which is good, since both the sender and I are in the central European time zone.

So it's 12:00 -> 13:00 CET(UTC+1) until end of march 2021 and in then it will still be 12:00 -> 13:00, just the daylight savings time will happen then, so CEST(UTC+2).

That "W. Europe Standard Time" thing is either a mistake on Microsoft's side or of the user who is sending me those

[lhca]

Thankfully calcurse ignores the TZID. AFAIK "W. Europe Standard Time" is what Britain and Portugal have, However the time zone info like TZOFFSET shows the central European time zone.

The central European one is also what is used when importing, which is good, since both the sender and I are in the central European time zone.

So it's 12:00 -> 13:00 CET(UTC+1) until end of march 2021 and in then it will still be 12:00 -> 13:00, just the daylight savings time will happen then, so CEST(UTC+2).

This is not what happens. The import file has

DTSTART;TZID=W. Europe Standard Time:20201029T110000
...
DTEND;TZID=W. Europe Standard Time:20201029T120000

which says

29 Oct 2020, 11:00 to 29 Oct 2020, 12:00 W. Europe Standard Time

The time zone specification "W. Europe Standard Time" is (probably) not recognized if you are on a Unix-like system. These systems use the tz database where time zones have names like Europe/Oslo (see @mvhulten's test file). Your system (probably) defaults to UTC, which at this time of the year is the same as W. Europe Standard TIme, and calcurse converts it to your local time: 12:00 - 13:00.

[mvhulten]

That time zone specification looks non-standard. RFC 5545 "does not define a naming convention for time zone identifiers.", but the use of the TZ database is ubiquitous throughout computer systems and does specify the formatting AREA/CITY (e.g. Europe/Oslo).

[lhca]

Ubiquitous? At least on Unix-like systems, presumably including MAC OS X. That makes the time zone component in your test file even stranger. An iCalendar time zone component is meant to specify the rules for a particular time zone, here Europe/Oslo. But there is no need for such a component if MAC OS uses the TZ database. Furthermore, the component is incomplete: in only says when winter (standard) time begins, not when it stops, i.e. when summer (daylight saving) time begins.

But in the case at hand all of this doesn't matter since calcurse ignores the time zone component completely and relies on the TZ database to convert from Europe/Oslo time to local time.

[lhca]

When import fails, like in this case, calcurse -i returns an error code of 0, so it's not easy to catch the error in a script.

@mvhulten, should calcurse distinguish between two cases only:

• all appointments, events and to-dos were imported, exit value 0
• at least one appointment, event or to-do was skipped, exit value 1

Or would it be useful to let calcurse distinguish between different kinds of import failure?

[mvhulten]

@lhca, I thought about this today, but I couldn't decide what is best.

An exit value of either 0 or 1 would suffice for me. If it fails on any of these, I will want to check the calcurse_log.* (which states the problem with line number of the ics file) and would manually add a to-do or whatever is missing and relevant to me. I wonder if there are good scenarios where more differentiation is useful.

Still, one way is to increase the exit code with +1 in case of failed appointment, +2 with failed event and +4 with failed to-do (or generalised +2^n starting from n=0).

[waynew]

It would also be useful if calcurse could provide a dump of the offending entries. I just discovered calcurse, and this RRULE funkiness.

But importing my whole calendar re-imports the whole calendar.