python-caldav / caldav

Apache License 2.0
330 stars 98 forks source link

test_create_ical can be flaky on Python 3.12 #380

Open slyon opened 9 months ago

slyon commented 9 months ago

Hello,

we're seeing some flakyness with the test_create_ical test case on Ubuntu's heavily loaded autopkgtest infrastructure. It seems like there's a timing issue, where test data can be generated at different points in time and the test fails when the data is not generated in the same second, see below:

299s E AssertionError: b'BEG[45 chars]163454Z\nDTSTART:20321010T101010Z\nDURATION:PT[96 chars]:2.0' != b'BEG[45 chars]163455Z\nDTSTART:20321010T101010Z\nDURATION:PT[96 chars]:2.0'

Notes: This is probably not related to https://github.com/python-caldav/caldav/issues/230 as we're seeing the issue on v0.11 There is also a deprecation warning when using python3.12, I don't know if that might have an impact:

299s   /usr/lib/python3/dist-packages/dateutil/tz/tz.py:37: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
299s     EPOCH = datetime.datetime.utcfromtimestamp(0)

Full logs (https://autopkgtest.ubuntu.com/results/autopkgtest-noble/noble/armhf/p/python-caldav/20240206_163515_8815a@/log.gz):

294s I: pybuild base:305: cd /tmp/autopkgtest.l1XavR/autopkgtest_tmp/build; python3.12 -m pytest tests
298s ============================= test session starts ==============================
298s platform linux -- Python 3.12.1, pytest-7.4.4, pluggy-1.3.0
298s rootdir: /tmp/autopkgtest.l1XavR/autopkgtest_tmp/build
298s collected 107 items
298s 
298s tests/test_caldav.py sssssssssssssssssssssssssssssssssssssssssssssssssss [ 47%]
298s ssssssssssssssssssssss                                                   [ 68%]
298s tests/test_caldav_unit.py ...s......................                     [ 92%]
298s tests/test_cdav.py ......                                                [ 98%]
298s tests/test_utils.py .                                                    [ 99%]
299s tests/test_vcal.py F                                                     [100%]
299s 
299s =================================== FAILURES ===================================
299s __________________________ TestVcal.test_create_ical ___________________________
299s 
299s self = <tests.test_vcal.TestVcal testMethod=test_create_ical>
299s 
299s     def test_create_ical(self):
299s         def create_and_validate(**args):
299s             return self.verifyICal(create_ical(**args))
299s     
299s         ## First, a fully valid ical_fragment should go through as is
299s         self.assertSameICal(create_and_validate(ical_fragment=ev), ev)
299s     
299s         ## One may add stuff to a fully valid ical_fragment
299s         self.assertSameICal(
299s             create_and_validate(ical_fragment=ev, priority=3), ev + "\nPRIORITY:3\n"
299s         )
299s     
299s         ## binary string or unicode string ... shouldn't matter
299s         self.assertSameICal(
299s             create_and_validate(ical_fragment=ev.encode("utf-8"), priority=3),
299s             ev + "\nPRIORITY:3\n",
299s         )
299s     
299s         ## The returned ical_fragment should always contain BEGIN:VCALENDAR and END:VCALENDAR
299s         ical_fragment = ev.replace("BEGIN:VCALENDAR", "").replace("END:VCALENDAR", "")
299s         self.assertSameICal(create_and_validate(ical_fragment=ical_fragment), ev)
299s     
299s         ## Create something with a dtstart and verify that we get it back in the ical
299s         some_ical0 = create_and_validate(
299s             summary="gobledok",
299s             dtstart=datetime(2032, 10, 10, 10, 10, 10, tzinfo=utc),
299s             duration=timedelta(hours=5),
299s         )
299s         some_ical1 = create_and_validate(
299s             summary=b"gobledok",
299s             dtstart=datetime(2032, 10, 10, 10, 10, 10, tzinfo=utc),
299s             duration=timedelta(hours=5),
299s         )
299s         assert re.search(b"DTSTART(;VALUE=DATE-TIME)?:20321010T101010Z", some_ical0)
299s >       self.assertSameICal(some_ical0, some_ical1, ignore_uid=True)
299s 
299s tests/test_vcal.py:103: 
299s _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
299s tests/test_vcal.py:48: in assertSameICal
299s     self.assertEqual(normalize(ical1, ignore_uid), normalize(ical2, ignore_uid))
299s E   AssertionError: b'BEG[45 chars]163454Z\nDTSTART:20321010T101010Z\nDURATION:PT[96 chars]:2.0' != b'BEG[45 chars]163455Z\nDTSTART:20321010T101010Z\nDURATION:PT[96 chars]:2.0'
299s =============================== warnings summary ===============================
299s ../../../../usr/lib/python3/dist-packages/dateutil/tz/tz.py:37
299s   /usr/lib/python3/dist-packages/dateutil/tz/tz.py:37: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
299s     EPOCH = datetime.datetime.utcfromtimestamp(0)
299s 
299s -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
299s =========================== short test summary info ============================
299s FAILED tests/test_vcal.py::TestVcal::test_create_ical - AssertionError: b'BEG...
299s ============= 1 failed, 32 passed, 74 skipped, 1 warning in 4.28s ==============
299s E: pybuild pybuild:391: test: plugin distutils failed with: exit code=1: cd /tmp/autopkgtest.l1XavR/autopkgtest_tmp/build; python3.12 -m pytest tests
tobixen commented 9 months ago

I've also observed things like this from time to time. I'm not sure when I get time to look into it, a pull request is appreciated.

tobixen commented 3 weeks ago

Probably the assertSameICal-method should be mended to accept minor deviations in DTSTAMP