ecederstrand / exchangelib

Python client for Microsoft Exchange Web Services (EWS)
BSD 2-Clause "Simplified" License
1.16k stars 249 forks source link

Can't browse public_folders_root anymore #1288

Closed davidemorellatobibit closed 4 months ago

davidemorellatobibit commented 4 months ago

A working code is no longer able to browse public_folders_root folder


from exchangelib.folders import Calendar from exchangelib import IMPERSONATION,DELEGATE,Credentials, Account, CalendarItem, UTC_NOW, Configuration, EWSDateTime from exchangelib.version import Version, EXCHANGE_O365 from exchangelib import OAuth2Credentials, OAUTH2

credentials = OAuth2Credentials ( client_id='omissis', client_secret='omissis', tenant_id='omissis' ) config=Configuration(server='outlook.office365.com', credentials=credentials, version=Version(build=EXCHANGE_O365), auth_type=OAUTH2 )

import datetime import json

import EWS_config

account = Account( primary_smtp_address=EWS_config.primary_smtp_address, autodiscover=False, config=config, access_type=IMPERSONATION )

account.public_folders_root.refresh() print(account.public_folders_root.tree())


Returns "IPM_SUBTREE" and no subfolders, that exist.

Python 3.7 and exchangelib 5.0.2 on debian 10

Thanks

ecederstrand commented 4 months ago

I believe this is what has been reported in https://github.com/ecederstrand/exchangelib/issues/1273#issuecomment-2018108441 as well.

TheyCallMeJames commented 4 months ago

we are getting similar issue as of around the same time yesterday. Our error says, "Not Allowed to access Non IPM Folder". This was working until yesterday.

ecederstrand commented 4 months ago

If this happened without changing the exchangelib version, then it must be caused either by a change in the server configuration or because of a server patch/upgrade.

joshuajung commented 4 months ago

This post from the Veeam Community Forum (an Exchange backup solution also affected) may help solve the issue: https://forums.veeam.com/veeam-backup-for-microsoft-365-f47/failed-to-get-folder-properties-not-allowed-to-access-non-ipm-folder-t93166.html#p515735:

The issue is caused by the hidden TeamsMessagesData folder that has been used in the past for journaling Teams messages. Until yesterday Microsoft has not blocked access to this folder after the access via EWS APIs was deprecated and the new paid APIs were introduced.

mattj-uss commented 4 months ago

I'm getting the "Not Allowed to access Non IPM Folder" error suddenly as well. It started at 12:48pm GMT-5. We use O365 Exchange.

ecederstrand commented 4 months ago

Can someone post the XML request leading to the reponse that contains the error message, and that response as well? See https://ecederstrand.github.io/exchangelib/#troubleshooting on capturing the XML.

Tre-Seibert commented 4 months ago

@ecederstrand

        account = Account(
            primary_smtp_address=session["email"],
            access_type=DELEGATE,
            config=conf,
            autodiscover=False,
        )

        print("TOP")
        print(account.public_folders_root.tree())
        print("bottom")

Request XML:

<?xml version='1.0' encoding='utf-8'?>
<s:Envelope
    xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
    xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
  <s:Header>
    <t:RequestServerVersion Version="Exchange2016"/>
    <t:TimeZoneContext>
      <t:TimeZoneDefinition Id="Eastern Standard Time"/>
    </t:TimeZoneContext>
  </s:Header>
  <s:Body>
    <m:GetFolder>
      <m:FolderShape>
        <t:BaseShape>IdOnly</t:BaseShape>
        <t:AdditionalProperties>
          <t:FieldURI FieldURI="folder:DisplayName"/>
          <t:FieldURI FieldURI="folder:FolderClass"/>
          <t:FieldURI FieldURI="folder:PermissionSet"/>
        </t:AdditionalProperties>
      </m:FolderShape>
      <m:FolderIds>
        <t:FolderId Id="************" ChangeKey="************"/>
        <t:FolderId Id="************" ChangeKey="************"/>
      </m:FolderIds>
    </m:GetFolder>
  </s:Body>
</s:Envelope>

Response XML:

<?xml version='1.0' encoding='utf-8'?>
<s:Envelope
    xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
  <s:Header>
    <h:ServerVersionInfo
    xmlns:h="http://schemas.microsoft.com/exchange/services/2006/types"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" MajorVersion="15" MinorVersion="20" MajorBuildNumber="7409" MinorBuildNumber="45" Version="V2018_01_08"/>
  </s:Header>
  <s:Body>
    <m:GetFolderResponse
    xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
      <m:ResponseMessages>
        <m:GetFolderResponseMessage ResponseClass="Success">
          <m:ResponseCode>NoError</m:ResponseCode>
          <m:Folders>
            <t:Folder>
              <t:FolderId Id="************" ChangeKey="************"/>
              <t:FolderClass>IPF.Note</t:FolderClass>
              <t:DisplayName>TB Ticket Archives</t:DisplayName>
              <t:PermissionSet>
                <t:Permissions>
                  <t:Permission>
                    <t:UserId>
                      <t:DistinguishedUser>Default</t:DistinguishedUser>
                    </t:UserId>
                    <t:CanCreateItems>false</t:CanCreateItems>
                    <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>None</t:EditItems>
                    <t:DeleteItems>None</t:DeleteItems>
                    <t:ReadItems>None</t:ReadItems>
                    <t:PermissionLevel>None</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-26622769</t:SID>
                      <t:PrimarySmtpAddress>************</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>All</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>PublishingEditor</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-49665204</t:SID>
                      <t:PrimarySmtpAddress>s************</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>None</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>Custom</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-49665143</t:SID>
                      <t:PrimarySmtpAddress>s************</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                    <t:IsFolderOwner>true</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>true</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>All</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>Owner</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-49665205</t:SID>
                      <t:PrimarySmtpAddress>************@************.com</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>None</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>Custom</t:PermissionLevel>
                  </t:Permission>
                </t:Permissions>
              </t:PermissionSet>
            </t:Folder>
          </m:Folders>
        </m:GetFolderResponseMessage>
        <m:GetFolderResponseMessage ResponseClass="Success">
          <m:ResponseCode>NoError</m:ResponseCode>
          <m:Folders>
            <t:Folder>
              <t:FolderId Id="************" ChangeKey="************"/>
              <t:DisplayName>TECHBLDRS INC</t:DisplayName>
              <t:PermissionSet>
                <t:Permissions>
                  <t:Permission>
                    <t:UserId>
                      <t:DistinguishedUser>Default</t:DistinguishedUser>
                    </t:UserId>
                    <t:CanCreateItems>false</t:CanCreateItems>
                    <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>None</t:EditItems>
                    <t:DeleteItems>None</t:DeleteItems>
                    <t:ReadItems>None</t:ReadItems>
                    <t:PermissionLevel>None</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-26622769</t:SID>
                      <t:PrimarySmtpAddress>************</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>All</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>PublishingEditor</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-49665143</t:SID>
                      <t:PrimarySmtpAddress>************@************.com</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                    <t:IsFolderOwner>true</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>true</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>All</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>Owner</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-49665204</t:SID>
                      <t:PrimarySmtpAddress>************@************.com</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>false</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>None</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>Custom</t:PermissionLevel>
                  </t:Permission>
                  <t:Permission>
                    <t:UserId>
                      <t:SID>S-1-5-21-4157035154-673357884-1224084594-49665205</t:SID>
                      <t:PrimarySmtpAddress>************@************.com</t:PrimarySmtpAddress>
                      <t:DisplayName>************</t:DisplayName>
                    </t:UserId>
                    <t:CanCreateItems>true</t:CanCreateItems>
                    <t:CanCreateSubFolders>true</t:CanCreateSubFolders>
                    <t:IsFolderOwner>false</t:IsFolderOwner>
                    <t:IsFolderVisible>true</t:IsFolderVisible>
                    <t:IsFolderContact>false</t:IsFolderContact>
                    <t:EditItems>All</t:EditItems>
                    <t:DeleteItems>None</t:DeleteItems>
                    <t:ReadItems>FullDetails</t:ReadItems>
                    <t:PermissionLevel>Custom</t:PermissionLevel>
                  </t:Permission>
                </t:Permissions>
              </t:PermissionSet>
            </t:Folder>
          </m:Folders>
        </m:GetFolderResponseMessage>
      </m:ResponseMessages>
    </m:GetFolderResponse>
  </s:Body>
</s:Envelope>
TOP
DEBUG:exchangelib.util:Session 34443 thread 123145566810112: Useful response from https://outlook.office365.com/EWS/Exchange.asmx
DEBUG:exchangelib.protocol:Server outlook.office365.com: Releasing session 34443
DEBUG:exchangelib.version:API version "Exchange2016" worked but server reports version "V2018_01_08". Using "Exchange2016"
DEBUG:exchangelib.folders.base:Fallback to class Folder (folder_class None, name TECHBLDRS INC)
IPM_SUBTREE
bottom
mattj-uss commented 4 months ago

@ecederstrand Here's another example from my environment:

credentials = OAuth2Credentials(
    client_id=client_id,
    client_secret=client_secret,
    tenant_id=tenant_id,
    identity=Identity(primary_smtp_address=primary_smtp_address),
)

config = Configuration(credentials=credentials, server=smtp_svr, auth_type=OAUTH2)

account = Account(
    primary_smtp_address=primary_smtp_address,
    credentials=credentials,
    config=config,
    access_type=IMPERSONATION,
)

email_folder = account.inbox / inbox_subfolder_name  # Error occurs here

request.xml.txt response.xml.txt

shawnlinxl commented 4 months ago

Just adding to what's been said above - we started to have the same issue at around 1pm US Eastern time.

pebberio commented 4 months ago

This error appears on almost all actions like:

BUT, if you are using FolderCollection, you will not get that error. Here is a dirty, tiny, working example: i hope it helps finding differences in the code base.

possible WORKAROUND

fav_folder = None
inbox_folders = FolderCollection(account=account, folders=[account.inbox])
for folder in inbox_folders.find_folders(depth="Shallow"):
    if folder.name == "MyFavouriteSubFolder":
        fav_folder = folder

for mail in fav_folder.all().only("subject", "sender", "datetime_received"):
    print(mail.sender.email_address)
mattj-uss commented 4 months ago

@ecederstrand I noticed in my response xml (see my previous comment) that Teams folders are being queried (e.g. TeamsMessages and TeamsMeetings). Assuming this is at the core of the issue, is there a way we can exclude those folders?

necrosaromx commented 4 months ago

@ecederstrand I'm posting here examples as well, this have the response "Not allowed to access Non IPM folder."

Request XML

<?xml version= '1.0' encoding= 'utf-8'?>\n
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
    xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
    <s:Header>
        <t:RequestServerVersion Version="Exchange2016" />
        <t:ExchangeImpersonation>
            <t:ConnectingSID>
                <t:PrimarySmtpAddress>replaced_address@replaced_domain</t:PrimarySmtpAddress>
            </t:ConnectingSID>
        </t:ExchangeImpersonation>
        <t:TimeZoneContext>
            <t:TimeZoneDefinition Id="****REPLACED****" />
        </t:TimeZoneContext>
    </s:Header>
    <s:Body>
        <m:GetFolder>
            <m:FolderShape>
                <t:BaseShape>IdOnly</t:BaseShape>
                <t:AdditionalProperties>
                    <t:FieldURI FieldURI="folder:DisplayName" />
                    <t:FieldURI FieldURI="folder:FolderClass" />
                    <t:FieldURI FieldURI="folder:PermissionSet" />
                </t:AdditionalProperties>
            </m:FolderShape>
            <m:FolderIds>
                <t:FolderId
                    Id="****REPLACED****"
                    ChangeKey="****REPLACED****" />

            </m:FolderIds>
        </m:GetFolder>
    </s:Body>
</s:Envelope>

Response XML

<?xml version="1.0" encoding="utf-8"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
        <h:ServerVersionInfo MajorVersion="15" MinorVersion="20" MajorBuildNumber="7409"
            MinorBuildNumber="45" Version="V2018_01_08"
            xmlns:h="http://schemas.microsoft.com/exchange/services/2006/types"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" />
    </s:Header>
    <s:Body>
        <m:GetFolderResponse xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
            <m:ResponseMessages>
                <m:GetFolderResponseMessage ResponseClass="Success">
                    <m:ResponseCode>NoError</m:ResponseCode>
                    <m:Folders>
                        <m:GetFolderResponseMessage ResponseClass="Error">
                    <m:MessageText>Not allowed to access Non IPM folder.</m:MessageText>
                    <m:ResponseCode>ErrorAccessDenied</m:ResponseCode>
                    <m:DescriptiveLinkKey>0</m:DescriptiveLinkKey>
                    <m:Folders />
                   <m:GetFolderResponseMessage ResponseClass="Error">
                    <m:MessageText>Not allowed to access Non IPM folder.</m:MessageText>
                    <m:ResponseCode>ErrorAccessDenied</m:ResponseCode>
                    <m:DescriptiveLinkKey>0</m:DescriptiveLinkKey>
                    <m:Folders />
                </m:GetFolderResponseMessage>
                    </m:Folders>
                </m:GetFolderResponseMessage>
            </m:ResponseMessages>
        </m:GetFolderResponse>
    </s:Body>
</s:Envelope>

I edited the message to keep the error lines.. it was too big..

<m:GetFolderResponseMessage ResponseClass="Error">
                    <m:MessageText>Not allowed to access Non IPM folder.</m:MessageText>
                    <m:ResponseCode>ErrorAccessDenied</m:ResponseCode>
                    <m:DescriptiveLinkKey>0</m:DescriptiveLinkKey>
                    <m:Folders />
                </m:GetFolderResponseMessage>
shawnlinxl commented 4 months ago

This error appears on almost all actions like:

  • account.inbox / 'SufFolderName'

  • account.msg_folder_root / 'SufFolderName'

  • account.inbox.walk()

  • and so on

BUT, if you are using FolderCollection, you will not get that error. Here is a dirty, tiny, working example: i hope it helps finding differences in the code base.

possible WORKAROUND


fav_folder = None

inbox_folders = FolderCollection(account=account, folders=[account.inbox])

for folder in inbox_folders.find_folders(depth="Shallow"):

    if folder.name == "MyFavouriteSubFolder":

        fav_folder = folder

for mail in fav_folder.all().only("subject", "sender", "datetime_received"):

    print(mail.sender.email_address)

This worked for us as a stop gap fix. Thanks!

Tre-Seibert commented 4 months ago

This error appears on almost all actions like:

  • account.inbox / 'SufFolderName'
  • account.msg_folder_root / 'SufFolderName'
  • account.inbox.walk()
  • and so on

BUT, if you are using FolderCollection, you will not get that error. Here is a dirty, tiny, working example: i hope it helps finding differences in the code base.

possible WORKAROUND

fav_folder = None
inbox_folders = FolderCollection(account=account, folders=[account.inbox])
for folder in inbox_folders.find_folders(depth="Shallow"):
    if folder.name == "MyFavouriteSubFolder":
        fav_folder = folder

for mail in fav_folder.all().only("subject", "sender", "datetime_received"):
    print(mail.sender.email_address)

Awesome this worked for us appreciate it!

kietatran00 commented 4 months ago

Hey! I'm attempting to use FolderCollection as a workaround, but having some issues understanding on how to implement it. Would you guys be able to explain how to add it in so that it avoids trying to access the Teams* folder?

Thank you!

mattj-uss commented 4 months ago

This error appears on almost all actions like:

* account.inbox / 'SufFolderName'

* account.msg_folder_root / 'SufFolderName'

* account.inbox.walk()

* and so on

BUT, if you are using FolderCollection, you will not get that error. Here is a dirty, tiny, working example: i hope it helps finding differences in the code base.

possible WORKAROUND

fav_folder = None
inbox_folders = FolderCollection(account=account, folders=[account.inbox])
for folder in inbox_folders.find_folders(depth="Shallow"):
    if folder.name == "MyFavouriteSubFolder":
        fav_folder = folder

for mail in fav_folder.all().only("subject", "sender", "datetime_received"):
    print(mail.sender.email_address)

Worked for us as well.

mattj-uss commented 4 months ago

Hey! I'm attempting to use FolderCollection as a workaround, but having some issues understanding on how to implement it. Would you guys be able to explain how to add it in so that it avoids trying to access the Teams* folder?

Thank you!

Can you provide an example of what isn't working?

andreit23 commented 4 months ago

This error appears on almost all actions like:

  • account.inbox / 'SufFolderName'
  • account.msg_folder_root / 'SufFolderName'
  • account.inbox.walk()
  • and so on

BUT, if you are using FolderCollection, you will not get that error. Here is a dirty, tiny, working example: i hope it helps finding differences in the code base.

possible WORKAROUND

fav_folder = None
inbox_folders = FolderCollection(account=account, folders=[account.inbox])
for folder in inbox_folders.find_folders(depth="Shallow"):
    if folder.name == "MyFavouriteSubFolder":
        fav_folder = folder

for mail in fav_folder.all().only("subject", "sender", "datetime_received"):
    print(mail.sender.email_address)

Worked for moving to different folders:

def move_item(item, folder_name):
    print(f"Attempting to move item {item.subject} to folder {folder_name}.")
    inbox_folders = FolderCollection(account=account, folders=[account.inbox])
    for folder in inbox_folders.find_folders(depth="Shallow"):
        if folder.name == folder_name:
            item.move(to_folder=folder)
            return
    print(f"Could not find folder {folder_name}")
adiazma commented 4 months ago

Hi, I am receiving this issue Not Allowed to access Non IPM Folder, but when I call inbox_folders.find_folders(depth="Shallow") I don't receive any folder, does anyone know if this will have a solution?

shawnlinxl commented 4 months ago

Hi, I am receiving this issue Not Allowed to access Non IPM Folder, but when I call inbox_folders.find_folders(depth="Shallow") I don't receive any folder, does anyone know if this will have a solution?

Are your folders under inbox? This only works if your folders are subfolders of the "Inbox" folder.

necrosaromx commented 4 months ago

The workaround worked for the EWS O365 integration in Palo alto XSOAR...

this is dirty code, but can help you those ingest incidents with this integration, while a patch from palo alto emerges.

def get_folder_by_path(self, path, account=None, is_public=False):     # pragma: no cover
        """
        Retrieve folder by path
        :param path: path of the folder
        :param account: account associated with the requested path
        :param is_public: is the requested folder public
        :return: exchangelib Folder
        """
        if account is None:
            account = self.get_account()
        demisto.info(f'que hay en account: {dir(account)}')
        # handle exchange folder id
        #if len(path) == FOLDER_ID_LEN:
        #    folders_map = account.root._folders_map
        #    if path in folders_map:
        #        return account.root._folders_map[path]
        #if is_public:
        #    folder_result = account.public_folders_root
        #elif path == "AllItems":
        #    folder_result = account.root
        #else:
        #    folder_result = account.inbox.parent  # Top of Information Store
        demisto.info("Asignando el floder inbox.parent")
        inbox_folders = FolderCollection(account=account, folders=[account.inbox])
        demisto.info(f"folder count: {len(inbox_folders)}")
        demisto.info(f"folder content: {inbox_folders}")
        demisto.info(f"folder dir: {dir(inbox_folders)}")
        for folder in inbox_folders.folders: #find_folders(depth="Shallow"):
            #demisto.info(f"folder: {repr(folder)}")

            demisto.info(f'folder name::: {folder.name}')
            if folder.name == "Inbox":
                folder_result = folder
                demisto.info(f"folder dir: {dir(folder)}")

        #folder_result = account.root #SVL
        path = path.replace("/", "\\")
        path = path.split("\\")
        for sub_folder_name in path:
            demisto.info(f"sub_folder_name {sub_folder_name}") #SVL
            #folder_filter_by_name = [
            #    x
            #    for x in folder_result.children
            #    if x.name.lower() == sub_folder_name.lower()
            #]
            #if len(folder_filter_by_name) == 0:
            #    raise Exception(f"No such folder {path}")
            #folder_result = folder_filter_by_name[0]

        return folder_result
kietatran00 commented 4 months ago

The workaround worked for the EWS O365 integration in Palo alto XSOAR...

this is dirty code, but can help you those ingest incidents with this integration, while a patch from palo alto emerges.

def get_folder_by_path(self, path, account=None, is_public=False):     # pragma: no cover
        """
        Retrieve folder by path
        :param path: path of the folder
        :param account: account associated with the requested path
        :param is_public: is the requested folder public
        :return: exchangelib Folder
        """
        if account is None:
            account = self.get_account()
        demisto.info(f'que hay en account: {dir(account)}')
        # handle exchange folder id
        #if len(path) == FOLDER_ID_LEN:
        #    folders_map = account.root._folders_map
        #    if path in folders_map:
        #        return account.root._folders_map[path]
        #if is_public:
        #    folder_result = account.public_folders_root
        #elif path == "AllItems":
        #    folder_result = account.root
        #else:
        #    folder_result = account.inbox.parent  # Top of Information Store
        demisto.info("Asignando el floder inbox.parent")
        inbox_folders = FolderCollection(account=account, folders=[account.inbox])
        demisto.info(f"folder count: {len(inbox_folders)}")
        demisto.info(f"folder content: {inbox_folders}")
        demisto.info(f"folder dir: {dir(inbox_folders)}")
        for folder in inbox_folders.folders: #find_folders(depth="Shallow"):
            #demisto.info(f"folder: {repr(folder)}")

            demisto.info(f'folder name::: {folder.name}')
            if folder.name == "Inbox":
                folder_result = folder
                demisto.info(f"folder dir: {dir(folder)}")

        #folder_result = account.root #SVL
        path = path.replace("/", "\\")
        path = path.split("\\")
        for sub_folder_name in path:
            demisto.info(f"sub_folder_name {sub_folder_name}") #SVL
            #folder_filter_by_name = [
            #    x
            #    for x in folder_result.children
            #    if x.name.lower() == sub_folder_name.lower()
            #]
            #if len(folder_filter_by_name) == 0:
            #    raise Exception(f"No such folder {path}")
            #folder_result = folder_filter_by_name[0]

        return folder_result

Exactly what I needed for my use case, thank you sir!

robinreckmann commented 4 months ago

Another way to fix cases like account.inbox / 'SubFolderName' is to use two forward slashes: account.inbox // 'SubFolderName'

pebberio commented 4 months ago

Another way to fix cases like account.inbox / 'SubFolderName' is to use two forward slashes: account.inbox // 'SubFolderName'

Awesome HotFix, a no brainer! This is even more simple as my FolderCollection workaround and should also work for most use cases until the core gets fixed.

arochfold commented 4 months ago

Another way to fix cases like account.inbox / 'SubFolderName' is to use two forward slashes: account.inbox // 'SubFolderName'

Worked for me in preliminary test just now. Literally only changed the /s to //s. I guess it has to do with the cache?

NicoDupont commented 4 months ago

It work for me too if I want only iterate over email in specific folder replace / by // exemple : account.root/'Haut de la banque d\'informations'/'0-ADDFILES_AUTO'/'INSTRUCTION' by account.root//'Haut de la banque d\'informations'//'0-ADDFILES_AUTO'//'INSTRUCTION'

but not work when I want iterate over folder/subfolder

jcalero commented 4 months ago

Good work finding workarounds, unfortunately in our case we need to browse folders in a path, so we use folder.parts. If anyone can think of a workaround for calls like folder.parent and folder.parts, I'm all ears.

ecederstrand commented 4 months ago

The problem here seems to be that O365 has some folder that it will return in a FindFolder request, but that throws ErrorAccessDenied in a GetFolder request. All the workarounds suggested here work because they avoid doing a GetFolder on these folders.

We may be able to provide a fix for exchangelib to handle this situation.

ecederstrand commented 4 months ago

But let's continue discussion of the ErrorAccessDenied issue in #1290. This issue is about not being able to browse public folders, which is a different bug.

ecederstrand commented 4 months ago

I think the issue of not being able to browse public folders is the same as https://github.com/ecederstrand/exchangelib/issues/1267 where it's not possible to browse the archive.

I'm closing this issue because the discussion warped into a discussion of #1290, and because #1267 is more on-point and has a workaround.

vitalogy commented 4 months ago

@ecederstrand We also had problems and opened a ticket for this. Here is the official statement from microsoft support and I can share it with you:

There has been a recent change on our side where we have explicitly blocked access to Teams data folders from EWS (access to these folders has never been supported, but we are now actively blocking the access).

The error “Not allowed to access Non IPM folder.” Is returned when a client attempts to access the Teams data folders (e.g. to list items or folders contained within). The error should be treated as an access denied error, and shouldn’t affect access to any other part of the mailbox.

You are using a third party Python library to integrate with EWS, and the issue will need to be fixed there. I suspect you’ll need to add a further error check here to ignore the new error (as currently any unexpected error will break the folder iteration): exchangelib/exchangelib/folders/roots.py at master · ecederstrand/exchangelib (github.com). There may be other areas in the code that need updating also (though that will need to be investigated by the developer of the library).

shah5488 commented 4 months ago

I am unable to download all the sentbox or inbox mails, meaning when I download the data and write to an excel I get handful of emails like 5-6 only downloaded for a particular day, while when I open the mail using webmail on my browser I see the mails count in 50-100 etc for a particular day

def scrape_folder(self, folder, start, end, parent_folder):
    try:

        today_mails = []
        folder_path = (
            self.csv_path
            + "//"
            + str(start)[:10]
            + "//"
            + parent_folder
            + "//"
            + folder.name
        )
        file_path = os.path.join(folder_path, "mail.xlsx")
        if not os.path.exists(file_path):
            # for email in folder.filter(
            #     datetime_received__range=(start, end)
            # ).order_by("-datetime_received"):
            for email in folder.filter(datetime_sent__range=(start, end)).order_by(
                "-datetime_sent"
            ):
                try:
                    self.logger.info(f"email-> {email}")
                    result = self.get_messageinfo(email, start, end, folder.name)
                    self.mailbox_data.append(result)
                    today_mails.append(result)
                    with open(self.csv_path + "//output.txt", "a") as file:
                        file.write(str(result) + "\n\n")
                except Exception as e:
                    self.logger.info("---------------------")
                    self.logger.info(
                        email.subject.encode("ascii", "ignore").decode("ascii")
                    )
                    self.logger.info(e)
                    self.logger.info("---------------------")

            os.makedirs(folder_path)
            # Create the file path

            df = DataFrame(
                today_mails,
                columns=[
                    "currenttime",
                    "mailbody",
                    "categories",
                    "subject",
                    "sender",
                    "receiver",
                    "importance",
                    "flag_status",
                    "receive_weekday",
                    "folder_name",
                    "Datetime_received",
                    "Status",
                    "Datetime_completed",
                    "complete_hours",
                    "execution_date",
                ],
            )
            df.to_excel(file_path, index=False)

    except Exception as e:
        print("---------------------")
        print(f"error in an email of folder {folder.name}")
        print(e)
        print("---------------------")