search

collective / icalendar

icalendar parser library for Python
https://icalendar.readthedocs.io
Other
988 stars 171 forks source link

[BUG] problem with timezone tests on NetBSD #731

Open 0-wiz-0 opened 6 days ago

0-wiz-0 commented 6 days ago

Describe the bug

When running the self tests for 6.0.1 on NetBSD 10.99.12/amd64, there are about 20 self test failures due to timezone issues.

To Reproduce

FAILED src/icalendar/tests/test_encoding.py::test_parses_event_with_non_ascii_tzid_issue_237[<lambda>1-pytz] - AssertionError: assert datetime.datetime(2017, 5, 11, 13, 30, tzinfo=<DstTzInfo '(UTC-03:00) Brasília' Brasília standard-1 day, 21:00:00 STD>) == datetime.datetime(2017, 5, 11, 13, 30, tzinfo=tzfile('/us...
FAILED src/icalendar/tests/test_encoding.py::test_parses_event_with_non_ascii_tzid_issue_237[<lambda>1-zoneinfo] - AssertionError: assert datetime.datetime(2017, 5, 11, 13, 30, tzinfo=<tzicalvtz '(UTC-03:00) Brasília'>) == datetime.datetime(2017, 5, 11, 13, 30, tzinfo=tzfile('/usr/share/zoneinfo/America/Sao_Paulo'))
FAILED src/icalendar/tests/test_examples.py::test_creating_calendar_with_unicode_fields[utc4-pytz] - AssertionError: assert b'BEGIN:VCALE...VCALENDAR\r\n' == b'BEGIN:VCALE...VCALENDAR\r\n'
FAILED src/icalendar/tests/test_examples.py::test_creating_calendar_with_unicode_fields[utc4-zoneinfo] - AssertionError: assert b'BEGIN:VCALE...VCALENDAR\r\n' == b'BEGIN:VCALE...VCALENDAR\r\n'
FAILED src/icalendar/tests/test_parsing.py::test_no_tzid_when_utc[utc4-pytz-date0-DTSTART:20120716T000000Z] - AssertionError: assert b'DTSTART:20120716T000000Z' in b'BEGIN:VEVENT\r\nDTSTART;TZID=:20120716T000000\r\nEND:VEVENT\r\n'
FAILED src/icalendar/tests/test_parsing.py::test_no_tzid_when_utc[utc4-pytz-date1-DTSTART:20211117T150915Z] - AssertionError: assert b'DTSTART:20211117T150915Z' in b'BEGIN:VEVENT\r\nDTSTART;TZID=:20211117T150915\r\nEND:VEVENT\r\n'
FAILED src/icalendar/tests/test_parsing.py::test_handles_unique_tzid[<lambda>1-pytz-issue_466_respect_unique_timezone] - AssertionError: assert datetime.datetime(2022, 10, 21, 20, 0, tzinfo=<DstTzInfo '/Europe/CUSTOM' CEST+2:00:00 DST>) == datetime.datetime(2022, 10, 21, 20, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Sto...
FAILED src/icalendar/tests/test_parsing.py::test_handles_unique_tzid[<lambda>1-pytz-issue_466_convert_tzid_with_slash] - AssertionError: assert datetime.datetime(2022, 10, 21, 20, 0, tzinfo=<DstTzInfo 'Europe/Stockholm' CEST+2:00:00 DST>) == datetime.datetime(2022, 10, 21, 20, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/S...
FAILED src/icalendar/tests/test_parsing.py::test_no_tzid_when_utc[utc4-zoneinfo-date0-DTSTART:20120716T000000Z] - AssertionError: assert b'DTSTART:20120716T000000Z' in b'BEGIN:VEVENT\r\nDTSTART;TZID=:20120716T000000\r\nEND:VEVENT\r\n'
FAILED src/icalendar/tests/test_parsing.py::test_no_tzid_when_utc[utc4-zoneinfo-date1-DTSTART:20211117T150915Z] - AssertionError: assert b'DTSTART:20211117T150915Z' in b'BEGIN:VEVENT\r\nDTSTART;TZID=:20211117T150915\r\nEND:VEVENT\r\n'
FAILED src/icalendar/tests/test_parsing.py::test_handles_unique_tzid[<lambda>1-zoneinfo-issue_466_respect_unique_timezone] - AssertionError: assert datetime.datetime(2022, 10, 21, 20, 0, tzinfo=<tzicalvtz '/Europe/CUSTOM'>) == datetime.datetime(2022, 10, 21, 20, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Stockholm'))
FAILED src/icalendar/tests/test_parsing.py::test_handles_unique_tzid[<lambda>1-zoneinfo-issue_466_convert_tzid_with_slash] - AssertionError: assert datetime.datetime(2022, 10, 21, 20, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Stockholm')) == datetime.datetime(2022, 10, 21, 20, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Stockho...
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-pytz-0-exception_date0-20120529T100000] - AssertionError: assert datetime.datetime(2012, 5, 29, 10, 0, tzinfo=<DstTzInfo 'Europe/Vienna' CEST+2:00:00 DST>) == datetime.datetime(2012, 5, 29, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-pytz-1-exception_date1-20120403T100000] - AssertionError: assert datetime.datetime(2012, 4, 3, 10, 0, tzinfo=<DstTzInfo 'Europe/Vienna' CEST+2:00:00 DST>) == datetime.datetime(2012, 4, 3, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-pytz-2-exception_date2-20120410T100000] - AssertionError: assert datetime.datetime(2012, 4, 10, 10, 0, tzinfo=<DstTzInfo 'Europe/Vienna' CEST+2:00:00 DST>) == datetime.datetime(2012, 4, 10, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-pytz-3-exception_date3-20120501T100000] - AssertionError: assert datetime.datetime(2012, 5, 1, 10, 0, tzinfo=<DstTzInfo 'Europe/Vienna' CEST+2:00:00 DST>) == datetime.datetime(2012, 5, 1, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-pytz-4-exception_date4-20120417T100000] - AssertionError: assert datetime.datetime(2012, 4, 17, 10, 0, tzinfo=<DstTzInfo 'Europe/Vienna' CEST+2:00:00 DST>) == datetime.datetime(2012, 4, 17, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-zoneinfo-0-exception_date0-20120529T100000] - AssertionError: assert datetime.datetime(2012, 5, 29, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) == datetime.datetime(2012, 5, 29, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-zoneinfo-1-exception_date1-20120403T100000] - AssertionError: assert datetime.datetime(2012, 4, 3, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) == datetime.datetime(2012, 4, 3, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-zoneinfo-2-exception_date2-20120410T100000] - AssertionError: assert datetime.datetime(2012, 4, 10, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) == datetime.datetime(2012, 4, 10, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-zoneinfo-3-exception_date3-20120501T100000] - AssertionError: assert datetime.datetime(2012, 5, 1, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) == datetime.datetime(2012, 5, 1, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
FAILED src/icalendar/tests/test_recurrence.py::test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-zoneinfo-4-exception_date4-20120417T100000] - AssertionError: assert datetime.datetime(2012, 4, 17, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) == datetime.datetime(2012, 4, 17, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))

One example of the detailed output of a test:

____________________________________________________ test_list_exdate_to_ical_is_inverse_of_from_ical[<lambda>1-zoneinfo-4-exception_date4-20120417T100000] ____________________________________________________

events = {'_parser': <bound method Component.from_ical of <class 'icalendar.cal.Event'>>, '_data_source_folder': PosixPath('/sc...07-16 12:06:38+00:00, Parameters({})), 'SUMMARY': vText(b'A Recurring event wi
th multiple exdates\\, one per line.')})}
i = 4, exception_date = datetime.datetime(2012, 4, 17, 10, 0), exception_date_ics = b'20120417T100000', in_timezone = <function <lambda> at 0x72b4819c3690>

    @pytest.mark.parametrize('i, exception_date, exception_date_ics', [
        (0, datetime(2012, 5, 29, 10, 0), b'20120529T100000'),
        (1, datetime(2012, 4, 3, 10, 0),  b'20120403T100000'),
        (2, datetime(2012, 4, 10, 10, 0), b'20120410T100000'),
        (3, datetime(2012, 5, 1, 10, 0),  b'20120501T100000'),
        (4, datetime(2012, 4, 17, 10, 0), b'20120417T100000')
    ])
    def test_list_exdate_to_ical_is_inverse_of_from_ical(events, i, exception_date, exception_date_ics, in_timezone):
        exdate = events.event_with_recurrence_exdates_on_different_lines['exdate']
>       assert exdate[i].dts[0].dt == in_timezone(exception_date, 'Europe/Vienna')
E       AssertionError: assert datetime.datetime(2012, 4, 17, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) == datetime.datetime(2012, 4, 17, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna'))
E        +  where datetime.datetime(2012, 4, 17, 10, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/Vienna')) = vDDDTypes(2012-04-17 10:00:00+02:00, Parameters({'TZID': 'Europe/Vienna'})).dt
E        +  and   datetime.datetime(2012, 4, 17, 10, 0, tzinfo=tzfile('/usr/share/zoneinfo/Europe/Vienna')) = <function <lambda> at 0x72b4819c3690>(datetime.datetime(2012, 4, 17, 10, 0), 'Europe/Vienna')

src/icalendar/tests/test_recurrence.py:41: AssertionError

Expected behavior

No errors.

Environment

Additional context

n/a

niccokunzmann commented 5 days ago

Thanks for reporting this. It seems we have an assumption that works on Debian:

zoneinfo.ZoneInfo(key='Europe/Vienna') == tzfile('/usr/share/zoneinfo/Europe/Vienna')

I guess, this just needs a different function to compare datetime for equality. That would fix the tests. Would you like to take this on?

One could also add another operating system in the CI if that is wished for. I do not know how otherwise we will be able to prevent this in the future.

0-wiz-0 commented 5 days ago

I don't feel comfortable tackling this, since I don't know the code at all. I tried playing around with Python and got this far:

>>> import zoneinfo
>>> from dateutil import tz
>>> zoneinfo.ZoneInfo(key='Europe/Vienna') == tz.tzfile('/usr/share/zoneinfo/Europe/Vienna')
False
>>> print(zoneinfo.ZoneInfo(key='Europe/Vienna'), tz.tzfile('/usr/share/zoneinfo/Europe/Vienna'))
Europe/Vienna tzfile('/usr/share/zoneinfo/Europe/Vienna')

Is that what the test code is currently doing? These objects look quite different to me :-)

The path to the zone file is the same on NetBSD:

-r--r--r--  1 root  wheel  658 Oct 12 11:13 /usr/share/zoneinfo/Europe/Vienna
niccokunzmann commented 4 days ago

I think, for now, I would assume that icalendar does work under that OS. However, the tests do not run because the timezones have a different module as origin.

(1) I think that there should be some way to track this down. (2) But we can also make tests more generic.

I like option 2.

E.g. this line can be changed:

assert exdate[i].dts[0].dt == in_timezone(exception_date, 'Europe/Vienna')

to 

assert_datetime_is_in_timezone(exdate[i].dts[0].dt, exception_date, 'Europe/Vienna')

and that makes sure that the datetime is equal and the timezone has the string 'Europe/Vienna' in it. I think, that is enough testing... identity of the timezones is not required but only an assumption here. We want the timezone object to be any 'Europe/Vienna' timezone and not necessarily be a ZoneInfo object.

What are your thoughts on this? If you need any help to get this done, I am here. Your contribution is much appreciated!