Tests are failing

[azmeuk]

The testTodoDatasearch fails on this test:

https://github.com/python-caldav/caldav/blob/1028f12b70c9553d0f33f3afce177ff44d729dd4/tests/test_caldav.py#L1498

        if (
            not self.check_compatibility_flag("no_expand")
            and not self.check_compatibility_flag("no_recurring")
            and not self.check_compatibility_flag("no_recurring_todo")
        ):
>           assert len([x for x in todos1 if "DTSTART:20020415T1330" in x.data]) == 1
E           assert 0 == 1
E            +  where 0 = len([])

The tests pass on GHA, I suppose because the tests are ignored due to:

https://github.com/python-caldav/caldav/blob/1028f12b70c9553d0f33f3afce177ff44d729dd4/tests/test_caldav.py#L1422-L1423

I am not sure why this is different on GHA and on my machine. I suppose this would be good if no test were ignored on GHA.

[tobixen]

At one hand I sort of regret that I wrote the testTodoDateSearch-thing, because it's not really testing the caldav library, it's actually testing the caldav servers - and few servers get everything "right" when it comes to date searches on tasks (and the standards are not very clear, so it may be different opinions out there on what is the right behaviour).

Anyway, if you've just been running pytest without any configuration in tests/conf_private.py, then the test should pass - unless your version of Radicale or Xandikos behaves differently than the version I'm using, and the version used by the Github infrastructure.

If you've set up the tests to go towards your server in tests/conf_private.py, then you can pass 'incompatibilities': ['no_todo_datesearch'] to have it skip this test. Have a look through tests/compatibility_issues.py to see the known compatibility issues.

[tobixen]

Edit: I see now that tests are failing on 3.11. That's interesting ... very interesting. I'll have a look into it before approving the pull request. (I bet that the problem is that xandikos or radicale is behaving differently in 3.11 than in 3.10)

[tobixen]

Edit: Now that I've drunk my morning coffee, I see that tests are failing at all python versions. Hence, an upgrade of a third part library (most likely xandikos or radicale) is causing the failure. I'll look into it ...

[tobixen]

I managed to reproduce the breakage by upgrading the recurring-ical-events library from 1.1.0b0 to 2.0.0

[tobixen]

The assumption that something was off in the calendar server may be broken ... Radicale does not use the recurring-ical-events library, but the caldav library does.

[azmeuk]

I have just tested the development branch and I still get the errors, notably on tests/test_caldav_unit.py::TestExpandRRule::test241. You can see this on the CI too.

This is stange because the version of recurring-ical-events installed on my virtualenv is 2.0.0

>       assert len(self.todo.data) > 128
E       AssertionError: assert 85 > 128
E        +  where 85 = len('BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nEND:VCALENDAR\n')
E        +    where 'BEGIN:VCALENDAR\nVERSION:2.0\nPRODID:-//Example Corp.//CalDAV Client//EN\nEND:VCALENDAR\n' = Todo(Todo: None).data
E        +      where Todo(Todo: None) = <tests.test_caldav_unit.TestExpandRRule object at 0x7efc6249c190>.todo

tests/test_caldav_unit.py:243: AssertionError
------------------------------------------------------------------------------------------------------------ Captured log setup -------------------------------------------------------------------------------------------------------------
ERROR    root:vcal.py:97 Ical data was modified to avoid compatibility issues
NoneType: None
ERROR    root:vcal.py:103 ---
+++
@@ -1,4 +1,3 @@
-
 BEGIN:VCALENDAR
 VERSION:2.0
 PRODID:-//Example Corp.//CalDAV Client//EN

[tobixen]

Thanks for pointing out, I'll have a second look

[azmeuk]

Strangely, this works on my computer when I use tox, but not when I use pytest directly. That would be enough for testing though. Thanks!

[tobixen]

It should pass with pytest also. Is it exactly the same breakage? And what version of recurring-ical-events are you using? Came to think that the fix I made won't work for recurring-ical-events version 1 ... I should specify that version 2 is needed in setup.py.

[tobixen]

(Now I managed to spread the work on this issue over three separated pull requests, sorry for all the noise. So much for my new poilcy on working more cleanly by enforcing a policy to make all commits to the master branch via github pull requests ...)

[azmeuk]

Is it exactly the same breakage?

It breaks on testTodoDatesearch, and on this test only:

        ## verify that "expand" works
        if (
            not self.check_compatibility_flag("no_expand")
            and not self.check_compatibility_flag("no_recurring")
            and not self.check_compatibility_flag("no_recurring_todo")
        ):
>           assert len([x for x in todos1 if "DTSTART:20020415T1330" in x.data]) == 1
E           assert 0 == 1
E            +  where 0 = len([])

And what version of recurring-ical-events are you using?

Pip freeze gives me recurring-ical-events==2.0.0

... sorry for all the noise ...

No problem :) Thank you for your hard work :heart:

[tobixen]

Could it be that pytest and tox is testing different revisions of caldav?

[azmeuk]

I just tested with a fresh environment, I compared the installed versions with pip freeze and this only shows minimal changes:

$ .tox/py311/bin/pip freeze > tox311.txt
$ which pip
/home/azmeuk/.local/share/virtualenvs/caldav-gepr/bin/pip
$ pip install --editable ".[test]"
...
$ pip freeze > dev.txt
$ diff dev.txt tox311.txt
5c5
< -e git+ssh://git@github.com/azmeuk/caldav.git@d5a61fc0653c3880971600b76f7cb0d9af5de827#egg=caldav
---
> caldav @ file:///home/azmeuk/dev/caldav/.tox/.tmp/package/1/caldav-0.12.0.dev0.zip
11d10
< exceptiongroup==1.1.0
34d32
< tomli==2.0.1

[tobixen]

Can you check with the current development branch?

[azmeuk]

Ah, new error :) This time this is on tests/test_caldav.py::TestLocalRadicale::testCreateEvent and this is the same error within and outside tox environments. The error on testTodoDatesearch have disappeared though:

self = Event(Event: http://localhost:5232/user1/pythoncaldav-test/3a7c37fc-9509-11ed-b71a-3497f625a28d.ics), retry_on_failure = False

    def _put(self, retry_on_failure=True):
        ## SECURITY TODO: we should probably have a check here to verify that no such object exists already
        r = self.client.put(
            self.url, self.data, {"Content-Type": 'text/calendar; charset="utf-8"'}
        )
        if r.status == 302:
            path = [x[1] for x in r.headers if x[0] == "location"][0]
        elif not (r.status in (204, 201)):
            if retry_on_failure:
                ## This looks like a noop, but the object may be "cleaned".
                ## See https://github.com/python-caldav/caldav/issues/43
                self.vobject_instance
                return self._put(False)
            else:
>               raise error.PutError(errmsg(r))
E               caldav.lib.error.PutError: PutError at '400 Bad Request
E
E               b'Bad Request'', reason no reason

caldav/objects.py:2037: PutError
------------------------------------------------------------------------------------------------------------- Captured log call -------------------------------------------------------------------------------------------------------------
WARNING  radicale:put.py:200 Bad PUT request on '/user1/pythoncaldav-test/3a7c37fc-9509-11ed-b71a-3497f625a28d.ics': dtstart
Traceback (most recent call last):
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/vobject/base.py", line 529, in __getattr__
    return self.contents[toVName(name)][0]
KeyError: 'dtstart'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/app/put.py", line 95, in prepare
    item.prepare()
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/__init__.py", line 470, in prepare
    self.time_range
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/__init__.py", line 459, in time_range
    self._time_range = find_time_range(
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/__init__.py", line 324, in find_time_range
    radicale_filter.visit_time_ranges(vobject_item, tag, range_fn, infinity_fn)
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/filter.py", line 249, in visit_time_ranges
    dtstart = child.dtstart.value
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/vobject/base.py", line 531, in __getattr__
    raise AttributeError(name)
AttributeError: dtstart
WARNING  radicale:put.py:200 Bad PUT request on '/user1/pythoncaldav-test/3a7c37fc-9509-11ed-b71a-3497f625a28d.ics': dtstart
Traceback (most recent call last):
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/vobject/base.py", line 529, in __getattr__
    return self.contents[toVName(name)][0]
KeyError: 'dtstart'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/app/put.py", line 95, in prepare
    item.prepare()
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/__init__.py", line 470, in prepare
    self.time_range
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/__init__.py", line 459, in time_range
    self._time_range = find_time_range(
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/__init__.py", line 324, in find_time_range
    radicale_filter.visit_time_ranges(vobject_item, tag, range_fn, infinity_fn)
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/radicale/item/filter.py", line 249, in visit_time_ranges
    dtstart = child.dtstart.value
  File "/home/azmeuk/.local/share/virtualenvs/caldav-gepr/lib/python3.10/site-packages/vobject/base.py", line 531, in __getattr__
    raise AttributeError(name)
AttributeError: dtstart

[tobixen]

So the old failure disappeared, that's all I need to know :-) I can see the above failure as well, work in progress. :-)

[azmeuk]

With 62e5f367de457977a99d0511e96656b754db6b8e everything is fine!

[tobixen]

This resolved the last problem ... my minor bugfix came with a typo. My fingers seems to be running in arbitrary patterns over the keyboard sometimes :-)

diff --git a/caldav/objects.py b/caldav/objects.py
index f5644f0..e32ca33 100644
--- a/caldav/objects.py
+++ b/caldav/objects.py
@@ -702,7 +702,7 @@ class Calendar(DAVObject):
         ):
             ## TODO: the ical_fragment code is not much tested
             if ical and not "ical_fragment" in ical_data:
-                ical_data["ical_fragmanet"] = ical
+                ical_data["ical_fragment"] = ical
             return vcal.create_ical(objtype=objtype, **ical_data)
         return ical