Unnable to access shared calendar without permission on the mailbox root folder

[hamarituc]

Describe the bug

Accessing a shared calendar is not possible if the accessing account has access permissions to the calendar only, but not to the root folder of the target mailbox.

To Reproduce

1. Create a shared mailbox with mail alias mail@domain
2. Add some calendar items
3. Add Permission for another user username to the calendar folder

Add-MailboxFolderPermission -Identity mailboxname:\Calendar -User username -AccessRights Reviewer

4. Try to access the calendar

import exchangelib

creds = exchangelib.Credentials(username = 'username', password = 'password')
config = exchangelib.Configuration(service_endpoint = 'https://server.domain/EWS/Exchange.asmx', credentials = creds, version = exchangelib.Version(build = exchangelib.Build(15, 2, 0, 0)))
account = exchangelib.Account(primary_smtp_address = 'mail@domain', autodiscover = False, config = config, access_type = exchangelib.DELEGATE)
account.identity.upn = 'upn@domain'

list(account.calendar.all())

Expected behavior

The calendar items should be returned,

Log output

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/virtenv/msxcalendar/lib/python3.11/site-packages/cached_property.py:70, in threaded_cached_property.__get__(self, obj, cls)
     68 try:
     69     # check if the value was computed before the lock was acquired
---> 70     return obj_dict[name]
     72 except KeyError:
     73     # if not, do the calculation and release the lock

KeyError: 'calendar'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
File ~/virtenv/msxcalendar/lib/python3.11/site-packages/cached_property.py:70, in threaded_cached_property.__get__(self, obj, cls)
     68 try:
     69     # check if the value was computed before the lock was acquired
---> 70     return obj_dict[name]
     72 except KeyError:
     73     # if not, do the calculation and release the lock

KeyError: 'root'

During handling of the above exception, another exception occurred:

ErrorAccessDenied                         Traceback (most recent call last)
Cell In[1], line 9
      6 account = exchangelib.Account(primary_smtp_address = 'mail@domain', autodiscover = False, config = config, access_type = exchangelib.DELEGATE)
      7 account.identity.upn = 'upn@domain'
----> 9 list(account.calendar.all())

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/cached_property.py:74, in threaded_cached_property.__get__(self, obj, cls)
     70     return obj_dict[name]
     72 except KeyError:
     73     # if not, do the calculation and release the lock
---> 74     return obj_dict.setdefault(name, self.func(obj))

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/account.py:254, in Account.calendar(self)
    248 @threaded_cached_property
    249 def calendar(self):
    250     # If the account contains a shared calendar from a different user, that calendar will be in the folder list.
    251     # Attempt not to return one of those. An account may not always have a calendar called "Calendar", but a
    252     # Calendar folder with a localized name instead. Return that, if it's available, but always prefer any
    253     # distinguished folder returned by the server.
--> 254     return self.root.get_default_folder(Calendar)

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/cached_property.py:74, in threaded_cached_property.__get__(self, obj, cls)
     70     return obj_dict[name]
     72 except KeyError:
     73     # if not, do the calculation and release the lock
---> 74     return obj_dict.setdefault(name, self.func(obj))

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/account.py:350, in Account.root(self)
    348 @threaded_cached_property
    349 def root(self):
--> 350     return Root.get_distinguished(account=self)

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/folders/roots.py:113, in RootOfHierarchy.get_distinguished(cls, account)
    111     raise ValueError(f"Class {cls} must have a DISTINGUISHED_FOLDER_ID value")
    112 try:
--> 113     return cls.resolve(
    114         account=account,
    115         folder=DistinguishedFolderId(
    116             id=cls.DISTINGUISHED_FOLDER_ID,
    117             mailbox=Mailbox(email_address=account.primary_smtp_address),
    118         ),
    119     )
    120 except MISSING_FOLDER_ERRORS:
    121     raise ErrorFolderNotFound(f"Could not find distinguished folder {cls.DISTINGUISHED_FOLDER_ID}")

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/folders/base.py:516, in BaseFolder.resolve(cls, account, folder)
    513 @classmethod
    514 def resolve(cls, account, folder):
    515     # Resolve a single folder
--> 516     folders = list(FolderCollection(account=account, folders=[folder]).resolve())
    517     if not folders:
    518         raise ErrorFolderNotFound(f"Could not find folder {folder!r}")

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/folders/collections.py:335, in FolderCollection.resolve(self)
    333 # Fetch all properties for the remaining folders of folder IDs
    334 additional_fields = self.get_folder_fields(target_cls=self._get_target_cls())
--> 335 yield from self.__class__(account=self.account, folders=resolveable_folders).get_folders(
    336     additional_fields=additional_fields
    337 )

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/folders/collections.py:403, in FolderCollection.get_folders(self, additional_fields)
    398 # Add required fields
    399 additional_fields.update(
    400     (FieldPath(field=BaseFolder.get_field_by_fieldname(f)) for f in self.REQUIRED_FOLDER_FIELDS)
    401 )
--> 403 yield from GetFolder(account=self.account).call(
    404     folders=self.folders,
    405     additional_fields=additional_fields,
    406     shape=ID_ONLY,
    407 )

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/services/get_folder.py:43, in GetFolder._elems_to_objs(self, elems)
     42 def _elems_to_objs(self, elems):
---> 43     for folder, elem in zip(self.folders, elems):
     44         if isinstance(elem, Exception):
     45             yield elem

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/services/common.py:287, in EWSService._chunked_get_elements(self, payload_func, items, **kwargs)
    285 for i, chunk in enumerate(chunkify(filtered_items, self.chunk_size), start=1):
    286     log.debug("Processing chunk %s containing %s items", i, len(chunk))
--> 287     yield from self._get_elements(payload=payload_func(chunk, **kwargs))

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/services/common.py:308, in EWSService._get_elements(self, payload)
    304 while True:
    305     try:
    306         # Create a generator over the response elements so exceptions in response elements are also raised
    307         # here and can be handled.
--> 308         yield from self._response_generator(payload=payload)
    309         # TODO: Restore session pool size on succeeding request?
    310         return

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/services/common.py:684, in EWSService._get_elements_in_response(self, response)
    659 """Take a list of 'SomeServiceResponseMessage' elements and return the elements in each response message that
    660 we want the service to return. With e.g. 'CreateItem', we get a list of 'CreateItemResponseMessage' elements
    661 and return the 'Message' elements.
   (...)
    681 :return: a generator of items as returned by '_get_elements_in_container()
    682 """
    683 for msg in response:
--> 684     container_or_exc = self._get_element_container(message=msg, name=self.element_container_name)
    685     if isinstance(container_or_exc, (bool, Exception)):
    686         yield container_or_exc

File ~/virtenv/msxcalendar/lib/python3.11/site-packages/exchangelib/services/common.py:606, in EWSService._get_element_container(self, message, name)
    604 # response_class == 'Error', or 'Success' and not 'NoError'
    605 try:
--> 606     raise self._get_exception(code=response_code, text=msg_text, msg_xml=msg_xml)
    607 except self.ERRORS_TO_CATCH_IN_RESPONSE as e:
    608     return e

ErrorAccessDenied: Der Zugriff wird verweigert. Überprüfen Sie die Anmeldeinformationen, und versuchen Sie es dann erneut., Der zugrunde liegende MAPI-Stream hat eine Ausnahme ausgelöst

Additional context

The code tries to access the root folder of the shared mailbox which should not be necessary. It worked in version 5.0. There was a bug in version 5.1 which resulted in routing the access to the mailbox of the accessing user instead of the shared mailbox (#1222). It doesn't work in 5.2. Thus i cannot determine if the bug was introduced between 5.0 and 5.1 or between 5.1 and 5.2.

[ecederstrand]

I'm pretty sure we've always tried to access the root folder before accessing the calendar. Unfortunately, I don't have access to a multi-account test setup, so I'm not able to reproduce this myself. Your best bet is to enable debug logging and comparing the XML requests and responses of a working and a non-working version of exchangelib. That would reveal what version 5.2 does differently.

[hamarituc]

Attached you will find the log comparing the working version (5.0.3) and the failing version (5.1.0). I am wondering why the newer version connects with Exchange 2016 API version, wheres the older one connects with Exchange 2019 version. The server is Exchange 2019. Maybe this is the root cause of the issue?

• 5.0.3-working.txt
• 5.2.0-failing.txt

[ecederstrand]

It's possible that this was fixed by https://github.com/ecederstrand/exchangelib/commit/f65079f372a35d09a3d4b3e929ecc53bd60b6fc1. Can you test it out?

[hamarituc]

It's possible that this was fixed by f65079f. Can you test it out?

I tested the commit. The Exception is now gone, but the search returns an empty result set. Attached you will find the logs

• 5.0.3-working.txt
• f65079f372a35d09a3d4b3e929ecc53bd60b6fc1-failing.txt

... and the XML requests and responses I extracted from it.

• failing-req1.txt
• failing-req2.txt
• failing-req3.txt
• failing-resp1.txt
• failing-resp2.txt
• failing-resp3.txt
• working-req1.txt
• working-req2.txt
• working-req3.txt
• working-req4.txt
• working-req5.txt
• working-resp1.txt
• working-resp2.txt
• working-resp3.txt
• working-resp4.txt
• working-resp5.txt

I think the error happens when querying the calendar folder, as the t:Mailbox-Entity is missing.

--- working-req4.txt    2024-04-19 09:18:14.404610015 +0200
+++ failing-req3.txt    2024-04-19 09:09:02.180618517 +0200
@@ -13,13 +13,7 @@
       </m:ItemShape>
       <m:IndexedPageItemView MaxEntriesReturned="100" Offset="0" BasePoint="Beginning"/>
       <m:ParentFolderIds>
-        <t:DistinguishedFolderId Id="calendar">
-          <t:Mailbox>
-            <t:EmailAddress>mail@server.domain</t:EmailAddress>
-            <t:RoutingType>SMTP</t:RoutingType>
-            <t:MailboxType>Mailbox</t:MailboxType>
-          </t:Mailbox>
-        </t:DistinguishedFolderId>
+        <t:DistinguishedFolderId Id="calendar"/>
       </m:ParentFolderIds>
     </m:FindItem>
   </s:Body>

[ecederstrand]

Thanks for the helpful debug output!

This may have been fixed by https://github.com/ecederstrand/exchangelib/commit/d9035d03960797f9277381845e4ec98f837798ea

[hamarituc]

Thanks for the helpful debug output!

This may have been fixed by d9035d0

I am happy to confirm this patch fixes the issue :tada:. Thank you very much for your work.