fabien-michel
commented
1 month ago Can you provide the ics of the event? Is there a difference in the system / container locale?
Closed sanieldalib closed 2 weeks ago
fabien-michel
commented
1 month ago Can you provide the ics of the event? Is there a difference in the system / container locale?
sanieldalib
commented
1 month ago Thank you @fabien-michel!
Here is the ics: https://calendar.google.com/calendar/ical/themarinas%40gmail.com/public/basic.ics
With the following code
from urllib.request import urlopen
from datetime import datetime, timezone
from icalendar import Calendar
import recurring_ical_events
with urlopen(
"https://calendar.google.com/calendar/ical/themarinas%40gmail.com/public/basic.ics"
) as ical_str:
calendar = Calendar.from_ical(ical_str.read())
parsed_cal = recurring_ical_events.of(calendar)
start_time = datetime.fromtimestamp(1722564000, timezone.utc)
end_time = datetime.fromtimestamp(1722567600, timezone.utc)
events = parsed_cal.between(start_time, end_time)
print(f"Found {len(events)} events")
print(events)I get the following output on macOS:
Found 3 events
[VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:35:35+00:00, Parameters({})), 'UID': vText('b'aogpprh4bolu8ckmop49ca6404@google.com''), 'CREATED': vDDDTypes(2014-03-31 19:26:50+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2015-07-31 02:23:26+00:00, Parameters({})), 'SEQUENCE': 1, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers for the Feast of St. Joseph''), 'TRANSP': vText('b'OPAQUE'')}, VALARM({'ACTION': vText('b'NONE''), 'TRIGGER': vDDDTypes(1976-04-01 00:55:45+00:00, Parameters({'VALUE': 'DATE-TIME'})), 'X-WR-ALARMUID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B''), 'UID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B'')})), VEVENT({'DTSTART': vDDDTypes(2024-08-02, Parameters({'VALUE': 'DATE'})), 'DTEND': vDDDTypes(2024-08-03, Parameters({'VALUE': 'DATE'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:35:35+00:00, Parameters({})), 'UID': vText('b'040000008200E00074C5B7101A82E0080000000080791746E0B7C801000000000000000010000000296A1C3ED541F040AE0829C0C67BD7E0''), 'CREATED': vDDDTypes(2008-05-17 12:41:40+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2023-02-21 06:29:03+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Feast of St Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')}, VALARM({'ACTION': vText('b'AUDIO''), 'TRIGGER': vDDDTypes(-1 day, 9:00:00, Parameters({})), 'X-WR-ALARMUID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'UID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'ATTACH': 'Chord', 'X-APPLE-DEFAULT-ALARM': vText('b'TRUE''), 'ACKNOWLEDGED': vText('b'20200802T001626Z'')})), VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:35:35+00:00, Parameters({})), 'UID': vText('b'3DF21382-D125-4051-98BA-AE1412967940''), 'CREATED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers: Feast of St. Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')})]However, running within the python:3.9.11 container yields the following:
Found 2 events
[VEVENT({'DTSTART': vDDDTypes(2024-08-02, Parameters({'VALUE': 'DATE'})), 'DTEND': vDDDTypes(2024-08-03, Parameters({'VALUE': 'DATE'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:47:01+00:00, Parameters({})), 'UID': vText('b'040000008200E00074C5B7101A82E0080000000080791746E0B7C801000000000000000010000000296A1C3ED541F040AE0829C0C67BD7E0''), 'CREATED': vDDDTypes(2008-05-17 12:41:40+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2023-02-21 06:29:03+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Feast of St Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')}, VALARM({'ACTION': vText('b'AUDIO''), 'TRIGGER': vDDDTypes(-1 day, 9:00:00, Parameters({})), 'X-WR-ALARMUID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'UID': vText('b'F02F9E2A-F0FB-4E4F-A9AB-9405F33BBA19''), 'ATTACH': 'Chord', 'X-APPLE-DEFAULT-ALARM': vText('b'TRUE''), 'ACKNOWLEDGED': vText('b'20200802T001626Z'')})), VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-07-30 18:47:01+00:00, Parameters({})), 'UID': vText('b'3DF21382-D125-4051-98BA-AE1412967940''), 'CREATED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2024-07-30 02:36:49+00:00, Parameters({})), 'SEQUENCE': 0, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers: Feast of St. Joseph the Carpenter''), 'TRANSP': vText('b'OPAQUE''), 'X-APPLE-TRAVEL-ADVISORY-BEHAVIOR': vText('b'AUTOMATIC'')})]Here is the Dockerfile I used:
FROM python:3.9.11
WORKDIR /code
COPY test.py /code/
RUN pip install recurring-ical-events~=2.2.3
CMD python3 test.pyHi @fabien-michel thank you for your response. Just wanted to check if this is enough information for you to potentially reproduce?
niccokunzmann
commented
1 month ago I would usually say that the ICS file should be attached to the issue as it could change over time. This is a very interesting case. I thought that pure Python code would not differ between OS. To solve this, your code is really useful to copy into a test. Then, we could run the tests under MacOS.
sanieldalib
commented
1 month ago @niccokunzmann @fabien-michel Here is the ICS file:
basic.ics.txt
Github does not access .ics so I added the txt extension.
niccokunzmann
commented
1 month ago @sanieldalib Could you check this? I copied your code and now, I get this output:
~/recurring-ical-events$ .tox/py39/bin/python test.py
Found 0 events
[]Code of test.py
from urllib.request import urlopen
from datetime import datetime, timezone
from icalendar import Calendar
import recurring_ical_events
with urlopen(
"https://github.com/user-attachments/files/16516741/basic.ics.txt"
) as ical_str:
calendar = Calendar.from_ical(ical_str.read())
parsed_cal = recurring_ical_events.of(calendar)
start_time = datetime.fromtimestamp(1722564000, timezone.utc)
end_time = datetime.fromtimestamp(1722567600, timezone.utc)
events = parsed_cal.between(start_time, end_time)
print(f"Found {len(events)} events")
print(events)
@sanieldalib, I opened #153 for this but I think the calendar might have changed as the output is different ...
sanieldalib
commented
1 month ago @niccokunzmann Sorry, the ics was updated from my first posting to only include the troublesome event within the specified timestamps.
Here is the output of your test on my macOS machine:
✗ python3 test.py
Found 1 events
[VEVENT({'DTSTART': vDDDTypes(2024-08-01 19:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTEND': vDDDTypes(2024-08-01 20:00:00-07:00, Parameters({'TZID': 'America/Los_Angeles'})), 'DTSTAMP': vDDDTypes(2024-08-07 00:03:26+00:00, Parameters({})), 'UID': vText('b'aogpprh4bolu8ckmop49ca6404@google.com''), 'CREATED': vDDDTypes(2014-03-31 19:26:50+00:00, Parameters({})), 'LAST-MODIFIED': vDDDTypes(2015-07-31 02:23:26+00:00, Parameters({})), 'SEQUENCE': 1, 'STATUS': vText('b'CONFIRMED''), 'SUMMARY': vText('b'Vespers for the Feast of St. Joseph''), 'TRANSP': vText('b'OPAQUE'')}, VALARM({'ACTION': vText('b'NONE''), 'TRIGGER': vDDDTypes(1976-04-01 00:55:45+00:00, Parameters({'VALUE': 'DATE-TIME'})), 'X-WR-ALARMUID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B''), 'UID': vText('b'0D3A9816-AC61-499A-A594-930AA281666B'')}))]vs in docker (python:3.9.11) :
Found 0 events
[] Hm. I am under MacOS and I cannot reproduce this.
niccokunzmann@Niccos-MacBook-Pro recurring-ical-events % tox -re py38
py38: remove tox env folder /Users/niccokunzmann/recurring-ical-events/.tox/py38
py38: install_deps> python -I -m pip install -r /Users/niccokunzmann/recurring-ical-events/requirements.txt -r /Users/niccokunzmann/recurring-ical-events/test-requirements.txt --pre
py38: commands[0]> pytest --basetemp=/Users/niccokunzmann/recurring-ical-events/.tox/py38/tmp
======================================================== test session starts ========================================================
platform darwin -- Python 3.8.2, pytest-8.3.2, pluggy-1.5.0
cachedir: .tox/py38/.pytest_cache
rootdir: /Users/niccokunzmann/recurring-ical-events
configfile: pyproject.toml
plugins: cov-5.0.0
collected 1272 items
test/test_after.py ....................................................................ss [ 5%]
test/test_at_function.py ............................................ [ 8%]
test/test_bad_rrule_format.py .... [ 9%]
test/test_convert_inputs.py ........................ [ 11%]
test/test_daylight_saving_time.py ........ [ 11%]
test/test_deleted_entries.py ........ [ 12%]
test/test_duration.py ................................................................ [ 17%]
test/test_end_before_start_event.py .... [ 17%]
test/test_event_values_and_edits.py ............................................. [ 21%]
test/test_examples.py ....................................................................................................... [ 29%]
............. [ 30%]
test/test_issue_101_select_components.py .................................................................... [ 35%]
test/test_issue_107_omitting_last_event.py .... [ 36%]
test/test_issue_113_period_in_rdate.py ........ [ 36%]
test/test_issue_117_until_before_dtstart.py .... [ 37%]
test/test_issue_128_only_first_event.py .......... [ 37%]
test/test_issue_148_ignored_exdate_in_higher_sequence.py ................................................................ [ 42%]
test/test_issue_15.py .... [ 43%]
test/test_issue_151_macos_linux_difference.py FFFF [ 43%]
test/test_issue_18_cancel_status.py ........................ [ 45%]
test/test_issue_20_exdate_ignored.py ........................................................................ [ 51%]
test/test_issue_27.py ........ [ 51%]
test/test_issue_28_timezone_with_z.py .... [ 51%]
test/test_issue_36_recurrence_id_format.py ............ [ 52%]
test/test_issue_4.py .................... [ 54%]
test/test_issue_44_day_event_reported_twice.py ............................ [ 56%]
test/test_issue_48_daylight.py ........................ [ 58%]
test/test_issue_48_dst.py ............ [ 59%]
test/test_issue_61.py .... [ 59%]
test/test_issue_62_moved_event.py ................................................ [ 63%]
test/test_issue_6_copy_subcomponents.py ........ [ 64%]
test/test_issue_7_datetime_and_date_start_stop.py .................... [ 65%]
test/test_issue_86_x_wr_timezone_but_no_tzid_in_dt.py .... [ 66%]
test/test_issue_97_simple_recurrent_todos_and_journals.py .................................... [ 68%]
test/test_keep_recurrence_attributes.py ............ [ 69%]
test/test_multiple_rrule.py .... [ 70%]
test/test_properties.py ........................ [ 72%]
test/test_rdate.py ................................................................ [ 77%]
test/test_readme.py . [ 77%]
test/test_recurrence_sequence_number.py .... [ 77%]
test/test_repeated_properties.py .... [ 77%]
test/test_repetitions_do_not_change.py .................... [ 79%]
test/test_simple_recurrent_events.py .................................... [ 82%]
test/test_single_events.py .................... [ 83%]
test/test_skip_bad_events.py ........ [ 84%]
test/test_time_arguments.py ........ [ 85%]
test/test_time_span_contains_event.py ............................................. [ 88%]
test/test_time_zones_differ.py .................................... [ 91%]
test/test_x_wr_timezone.py ......................................................... [ 95%]
test/test_zero_size_events.py .................................... [ 98%]
test/test_zoneinfo_issue_57.py ................ [100%]
============================================================= FAILURES ==============================================================
_________________________________________ test_count_events_from_issue[Calendars-use_pytz] __________________________________________
calendars = Calendars(use_pytz)
def test_count_events_from_issue(calendars):
"""The number of events seems to differ from OS to OS."""
start_time = datetime.fromtimestamp(1722564000, timezone.utc)
end_time = datetime.fromtimestamp(1722567600, timezone.utc)
events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
for event in events:
print(event["UID"], event["DTSTART"], event["SUMMARY"])
> assert len(events) == 4
E assert 0 == 4
E + where 0 = len([])
test/test_issue_151_macos_linux_difference.py:17: AssertionError
_______________________________________ test_count_events_from_issue[Calendars-use_zoneinfo] ________________________________________
calendars = Calendars(use_zoneinfo)
def test_count_events_from_issue(calendars):
"""The number of events seems to differ from OS to OS."""
start_time = datetime.fromtimestamp(1722564000, timezone.utc)
end_time = datetime.fromtimestamp(1722567600, timezone.utc)
events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
for event in events:
print(event["UID"], event["DTSTART"], event["SUMMARY"])
> assert len(events) == 4
E assert 0 == 4
E + where 0 = len([])
test/test_issue_151_macos_linux_difference.py:17: AssertionError
___________________________________ test_count_events_from_issue[ReversedCalendars-use_zoneinfo] ____________________________________
calendars = ReversedCalendars(use_zoneinfo)
def test_count_events_from_issue(calendars):
"""The number of events seems to differ from OS to OS."""
start_time = datetime.fromtimestamp(1722564000, timezone.utc)
end_time = datetime.fromtimestamp(1722567600, timezone.utc)
events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
for event in events:
print(event["UID"], event["DTSTART"], event["SUMMARY"])
> assert len(events) == 4
E assert 0 == 4
E + where 0 = len([])
test/test_issue_151_macos_linux_difference.py:17: AssertionError
_____________________________________ test_count_events_from_issue[ReversedCalendars-use_pytz] ______________________________________
calendars = ReversedCalendars(use_pytz)
def test_count_events_from_issue(calendars):
"""The number of events seems to differ from OS to OS."""
start_time = datetime.fromtimestamp(1722564000, timezone.utc)
end_time = datetime.fromtimestamp(1722567600, timezone.utc)
events = calendars.issue_151_macos_linux_difference.between(start_time, end_time)
for event in events:
print(event["UID"], event["DTSTART"], event["SUMMARY"])
> assert len(events) == 4
E assert 0 == 4
E + where 0 = len([])
test/test_issue_151_macos_linux_difference.py:17: AssertionError
====================================================== short test summary info ======================================================
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[Calendars-use_pytz] - assert 0 == 4
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[Calendars-use_zoneinfo] - assert 0 == 4
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[ReversedCalendars-use_zoneinfo] - assert 0 == 4
FAILED test/test_issue_151_macos_linux_difference.py::test_count_events_from_issue[ReversedCalendars-use_pytz] - assert 0 == 4
============================================ 4 failed, 1266 passed, 2 skipped in 20.76s =============================================
py38: exit 1 (25.50 seconds) /Users/niccokunzmann/recurring-ical-events> pytest --basetemp=/Users/niccokunzmann/recurring-ical-events/.tox/py38/tmp pid=780
py38: FAIL code 1 (40.53=setup[15.03]+cmd[25.50] seconds)
evaluation failed :( (40.65 seconds)
niccokunzmann@Niccos-MacBook-Pro recurring-ical-events % This is in #153.
niccokunzmann
commented
1 month ago Could you run this command in the environment? pip list
niccokunzmann
commented
1 month ago Also, could you follow the setup documentation and run the tests on your machine? I am wondering where and how this problem occurs. I cannot reproduce it at the moment.
sanieldalib
commented
1 month ago @niccokunzmann I dug into this some more. It appears that the issue is a regression between versions 2.2.2 and 2.2.3. The event appears on any OS in version 2.2.2, but once I upgrade to 2.2.3, the event no longer appears.
I initially thought this was an issue with the OS as my Docker image, was pulling the latest patch version, which my venv on macOS already had v2.2.2 installed. I apologize for the red herring here.
niccokunzmann
commented
4 weeks ago No problem, this can happen. Would you say that the issue is still open and needs work on it? What as required to close it?
niccokunzmann
commented
4 weeks ago No problem, this can happen. Would you say that the issue is still open and needs work on it? What as required to close it?
sanieldalib
commented
4 weeks ago Yes. There is still a regression from 2.2.2 -> 2.2.3. The test you wrote should return 1 event, not 0.
niccokunzmann
commented
2 weeks ago This is fixed in v3.0.0.
This took me 12 hours - 720$ - as an invitation if you or someone else sees the importance to contribute to my FOSS work :)
I am running into an issue where the same code:
returns different events on macOS compared to a linux docker container (
python:3.9.11). Specifically, it is an annually recurring event missing when running within the linux container (python:3.9.11). I have confirmed that the difference is between OSes, as I get the difference in behavior on the same machine, python version, and package version - the only difference is macOS vs the docker image.I'll try to create a minimum reproduction of this and include it.
Is there anything you might be aware of that would be causing this issue?
We're using Polar.sh so you can upvote and help fund this issue. We receive the funding once the issue is completed & confirmed by you. Thank you in advance for helping prioritize & fund our work.