OAuth2Credentials requires SOAP impersonation headers unconditionally

I'm trying to connect to an email using Microsoft exchange with this library. Here is the code I'm using :

from exchangelib import Account, FileAttachment, ItemAttachment, Message, 
OAUTH2, Configuration, OAuth2Credentials

client_id = "xxxx"
client_secret = "xxxx"
tenant_id = "xxxx"
address = "myemail@domain.com"

credentials = OAuth2Credentials(
    client_id=client_id,
    client_secret=client_secret,
    tenant_id=tenant_id
)

config = Configuration(
    credentials=credentials,
    auth_type=OAUTH2
)

account = Account(primary_smtp_address=address, autodiscover=True, config=config)

for item in account.inbox.all():
    print("got item ", item.sender)

When I put in my credentials and run this program I get the following error:

Traceback (most recent call last):
  File "C:/Users/axk7812/Documents/Work/general-data/voip/run.py", line 42, in <module>
    email_3()
  File "C:/Users/axk7812/Documents/Work/general-data/voip/run.py", line 22, in email_3
    account = Account(primary_smtp_address=address, autodiscover=True, config=config)
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\account.py", line 85, in __init__
    self.ad_response, self.protocol = discover(
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\autodiscover\discovery.py", line 23, in discover
    return Autodiscovery(
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\autodiscover\discovery.py", line 99, in discover
    ad_response = self._step_1(hostname=domain)
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\autodiscover\discovery.py", line 329, in _step_1
    is_valid_response, ad = self._attempt_response(url=url)
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\autodiscover\discovery.py", line 289, in _attempt_response
    ad_protocol = AutodiscoverProtocol(
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\protocol.py", line 73, in __init__
    self._session_pool = self._create_session_pool()
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\protocol.py", line 160, in _create_session_pool
    session_pool.put(self.create_session(), block=False)
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\protocol.py", line 237, in create_session
    session = self.create_oauth2_session()
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\protocol.py", line 265, in create_oauth2_session
    raise ValueError('Auth type must be %r for credentials type OAuth2Credentials' % OAUTH2)
ValueError: Auth type must be 'OAuth 2.0' for credentials type OAuth2Credentials

Process finished with exit code 1

Somehow exchangelib is telling me I'm not setting OAuth but if you look in my Configuration setup I am setting it there. Am I doing something wrong here?

Tested this on both Windows 10 and Amazon Linux AMI 2018.03 (rhel fedora) using Python 3.8.1, and Exchangelib 3.1.1

Additionally - I've tried a separate way. Because I read in the comments of Configuration object class "If you want to use autodiscover, don't use a Configuration object." which seems necessary for setting the OAUTH2 flag because the only place to use it is in the Configuration class. So to use Auth2.0 is it necessary to turn off auto discover?

When I replace the bottom half of my code (Below credential definition)

    credentials = OAuth2Credentials(
        client_id=client_id,
        client_secret=client_secret,
        tenant_id=tenant_id
    )

    config = Configuration(
        credentials=credentials,
        service_endpoint="https://outlook.office365.com/EWS/Exchange.asmx",
        auth_type=OAUTH2
    )

    account = Account(primary_smtp_address=address, config=config)

    for item in account.inbox.all():
        print("got item ", item.sender)

I get a different error.

EWS https://outlook.office365.com/EWS/Exchange.asmx, account None: Exception in _get_elements: Traceback (most recent call last):
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\services\common.py", line 66, in _get_elements
    response = self._get_response_xml(payload=payload)
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\services\common.py", line 151, in _get_response_xml
    res = self._get_soap_messages(body=body, **parse_opts)
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\services\common.py", line 258, in _get_soap_messages
    cls._raise_soap_errors(fault=fault)  # Will throw SOAPError or custom EWS error
  File "C:\Users\axk7812\AppData\Local\Continuum\anaconda3\envs\voip\lib\site-packages\exchangelib\services\common.py", line 294, in _raise_soap_errors
    raise vars(errors)[code](msg)
exchangelib.errors.ErrorInvalidExchangeImpersonationHeaderData: ExchangeImpersonation SOAP header must be present for this type of OAuth token.

Not even sure how to set this SOAP header.

For additional reference - I tried these exact same credentials with an C# example in a different library so I think my account is setup correctly for exchange services.

OAuth2 hasn't been tested with autodiscover, I think. This is contrib code, so I'm not very familiar with how OAuth works with EWS myself.

The error you are getting in non-autodiscover mode can probably be solved by adding access_type=IMPERSONATION as argument when you create your Account.

Has there been any progress on this? I am getting the SOAP error as well. I am using the access_type=IMPERSONATION.

Here is the code that I am using:

client_id = 'XXXXX'
client_secret = 'XXXXX'
tenant_id = 'XXXXX'
address = 'XXXXX@XXXXX.com'

logging.basicConfig(level=logging.DEBUG, handlers=[PrettyXmlHandler()])

oauth_creds = OAuth2Credentials(client_id=client_id,
                                client_secret=client_secret,
                                tenant_id=tenant_id)
config = Configuration( credentials=oauth_creds,
                        service_endpoint='https://outlook.office365.com/EWS/Exchange.asmx',
                        auth_type=OAUTH2)
account = Account( primary_smtp_address=address,
                    autodiscover=False,
                    config=config,
                    access_type=IMPERSONATION)

for item in account.inbox.all():
    print("got item ", item.sender)

And this is the error I am getting:

WARNING:exchangelib.services.common:EWS https://outlook.office365.com/EWS/Exchange.asmx, account None: Exception in _get_elements: Traceback (most recent call last):
  File "/Users/x/Library/Python/2.7/lib/python/site-packages/exchangelib/services/common.py", line 71, in _get_elements
    response = self._get_response_xml(payload=payload)
  File "/Users/x/Library/Python/2.7/lib/python/site-packages/exchangelib/services/common.py", line 154, in _get_response_xml
    res = self._get_soap_messages(body=body, **parse_opts)
  File "/Users/x/Library/Python/2.7/lib/python/site-packages/exchangelib/services/common.py", line 261, in _get_soap_messages
    cls._raise_soap_errors(fault=fault)  # Will throw SOAPError or custom EWS error
  File "/Users/x/Library/Python/2.7/lib/python/site-packages/exchangelib/services/common.py", line 297, in _raise_soap_errors
    raise vars(errors)[code](msg)
ErrorInvalidExchangeImpersonationHeaderData: ExchangeImpersonation SOAP header must be present for this type of OAuth token.

Can you enable debug logging and post the contents of the SOAP header in the request that triggers this error response? See https://github.com/ecederstrand/exchangelib#troubleshooting on enabling logging.

Also, this could be related to https://github.com/ecederstrand/exchangelib/issues/743 if the version of Exchange expects an SmtpAddress instead of a PrimarySmtpAddress element.

I think that this is the SOAP header... Got a little lost in the logs! Let me know if you need anything else!

Thread: 4517651904 Auth type: <requests_oauthlib.oauth2_auth.OAuth2 object at 0x104f77e90> URL: https://outlook.office365.com/EWS/Exchange.asmx HTTP adapter: <requests.adapters.HTTPAdapter object at 0x104f77550> Allow redirects: False Streaming: False Response time: 0.510840892792 Status code: 500 Request headers: {'Content-Length': '479', 'Accept-Encoding': u'gzip, deflate', 'Accept': '*/*', 'User-Agent': u'exchangelib/2.2.0 (python-requests/2.22.0)', 'Connection': 'keep-alive', 'Cookie': 'exchangecookie=57823479d5bb4f2c91dbd7dc467ee2af', 'Content-Type': u'text/xml; charset=utf-8', u'Authorization': u'Bearer eyJ0eXAiOiJKV1QiLCJub25jZSI6ImxzUDlKaFJ6QWxIOXpYaEhqMk1Tb0x4QnRhaG1nU2Z2dG5CSGQzb1ZucmciLCJhbGciOiJSUzI1NiIsIng1dCI6IllNRUxIVDBndmIwbXhvU0RvWWZvbWpxZmpZVSIsImtpZCI6IllNRUxIVDBndmIwbXhvU0RvWWZvbWpxZmpZVSJ9.eyJhdWQiOiJodHRwczovL291dGxvb2sub2ZmaWNlMzY1LmNvbSIsImlzcyI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0LzRjNWZhOTU5LWFlNmUtNDZlZC1hODUyLTRlZTk4ZjBkODViZS8iLCJpYXQiOjE1ODY5NzIzMDgsIm5iZiI6MTU4Njk3MjMwOCwiZXhwIjoxNTg2OTc2MjA4LCJhaW8iOiI0MmRnWVBpL1ozV1UvZ3FGbG5rSFMrS1hiNXV2Q2dBPSIsImFwcF9kaXNwbGF5bmFtZSI6IkxMQiAtIEVXUyBPQXV0aCBBUEkgQWNjZXNzIiwiYXBwaWQiOiJiNTJkZTFhZC1mMzEwLTQxNTQtYjJlMS02NDU4ZjUwMTUxNDQiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC80YzVmYTk1OS1hZTZlLTQ2ZWQtYTg1Mi00ZWU5OGYwZDg1YmUvIiwib2lkIjoiZjA4ZjkyOWMtNzBhNi00OTg5LTllZmEtYmJhNTliNDA3M2EwIiwicm9sZXMiOlsiZnVsbF9hY2Nlc3NfYXNfYXBwIl0sInNpZCI6IjIwOGU2ZTc1LWM1ZmItNDg0Ny05YTMyLTVhNjc4MDUyNjdmOSIsInN1YiI6ImYwOGY5MjljLTcwYTYtNDk4OS05ZWZhLWJiYTU5YjQwNzNhMCIsInRpZCI6IjRjNWZhOTU5LWFlNmUtNDZlZC1hODUyLTRlZTk4ZjBkODViZSIsInV0aSI6Ik9tRm1FSHNYdkVpZncxVlByUUFBQUEiLCJ2ZXIiOiIxLjAifQ.GAknh5ftUuHXAoOpi4aGb8t4kbPx60oZkErvZHDBUbmYsmcSH5QQgJUxllMuc4pYpNMV7HXZMKS-Vm6G7qc0F3BZlgYmTVt8W1V7Rowv0CZSesdikb3fSGNYwBXGiZ7-nAcWiJi302lJTiRoFENFikkH1SB84v2yJ3hBdLQ-abGhMVfHyNbst_lcR2TM_J5FDXfDViHdOi1jYQDBR0CJTzA9YALnsbkPk3QUdCr4UnMUZPyMC1TdVExvPFYIEkHg3x7p-zDVyNu3D4feI9i4eNwn7sUre_kB_uXAomVVfdoqIPW0tX2_gwf-iIGraYqioiI3m-t3bORcBV3-ZfcWiQ'} Response headers: {'X-RUM-Validated': '1', 'Content-Length': '743', 'X-Proxy-RoutingCorrectness': '1', 'X-CalculatedBETarget': 'BN6PR10MB1876.namprd10.prod.outlook.com', 'Set-Cookie': 'exchangecookie=57823479d5bb4f2c91dbd7dc467ee2af; path=/; secure; SameSite=None', 'X-AspNet-Version': '4.0.30319', 'X-BEServer': 'BN6PR10MB1876', 'request-id': 'df63fe29-693a-4c84-bf70-b0cb47a61958', 'X-Powered-By': 'ASP.NET', 'x-ms-appId': 'b52de1ad-f310-4154-b2e1-6458f5015144', 'X-BeSku': 'Gen9', 'Server': 'Microsoft-IIS/10.0', 'X-Proxy-BackendServerStatus': '500', 'X-CalculatedFETarget': 'BN8PR16CU001.internal.outlook.com', 'X-DiagInfo': 'BN6PR10MB1876', 'X-FEServer': 'BN8PR16CA0026, MN2PR17CA0024', 'Cache-Control': 'private', 'Date': 'Wed, 15 Apr 2020 17:43:28 GMT', 'Content-Type': 'text/xml; charset=utf-8', 'X-BackEndHttpStatus': '500, 500', 'X-FEProxyInfo': 'BN8PR16CA0026.NAMPRD16.PROD.OUTLOOK.COM'} Request data: <?xml version='1.0' encoding='utf-8'?> <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><s:Header><t:RequestServerVersion Version="Exchange2016"/></s:Header><s:Body><m:ResolveNames ReturnFullContactData="false"><m:UnresolvedEntry>b52de1ad-f310-4154-b2e1-6458f5015144</m:UnresolvedEntry></m:ResolveNames></s:Body></s:Envelope> Response data: <?xml version="1.0" encoding="utf-8"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><s:Fault><faultcode xmlns:a="http://schemas.microsoft.com/exchange/services/2006/types">a:ErrorInvalidExchangeImpersonationHeaderData</faultcode><faultstring xml:lang="en-US">ExchangeImpersonation SOAP header must be present for this type of OAuth token.</faultstring><detail><e:ResponseCode xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ErrorInvalidExchangeImpersonationHeaderData</e:ResponseCode><e:Message xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ExchangeImpersonation SOAP header must be present for this type of OAuth token.</e:Message></detail></s:Fault></s:Body></s:Envelope>

Ok. I think this happens during version guessing. The last stack trace you posted doesn't contain enough steps to confirm this. Can you post a full stack trace - a trace that starts in your own code?

If this is indeed raised during version guessing, it's a bug in exchangelib. It's not going to be easy to fix. The root problem is that some services, e.g. ResolveNames, don't require account information, but account information is needed to construct the impersonation SOAP headers.

Anyway, I think you can work around this by supplying a version argument for your Configuration object. Something like this should do for O365:

from exchangelib.version import Version, EXCHANGE_O365

config = Configuration(
    ...,
    version=Version(build=EXCHANGE_O365),
)

All services accessible via account.protocol will still be broken when you are using this specific OAuth authentication mechanism, until this bug is fixed.

Adding the above Version code was able to fix the error I was having! Thanks so much!!

In the referenced commit, I added a new optional identity arg to OAuth2Credentials:

form exchangelib import Identity

OAuth2Credentials(
    client_id=client_id,
    client_secret=client_secret,
    tenant_id=tenant_id,
    identity=Identity(primary_smtp_address=..., smtp_address=..., upn=..., sid=...),
)

With this, you can specify either the SMTP address, UPN or SID of the account that the OAuth credentials were created for. This information will then go in the SOAP headers, which should fix the ErrorInvalidExchangeImpersonationHeaderData error you were seeing.

Closing. Feel free to reopen if you still have issues with this.

@ecederstrand I'm trying to connect email with this library using OAuth2 on Microsoft Exchange. It works properly when we pass the appropriate valid primary_smtp_address (existing email), but it does not return any response or error when we pass an invalid email address. my sample code is here :


import time
from exchangelib import Account, OAuth2Credentials, Configuration, OAUTH2, Identity

client_id='client_id'
client_secret='client_secret'
tenant_id='tenant_id'
user='email_id'
credentials = OAuth2Credentials(
    client_id=client_id,
    client_secret=client_secret,
    tenant_id=tenant_id,
    identity=Identity(smtp_address=user)
)

print(credentials)
config = Configuration(
    credentials=credentials,
    auth_type=OAUTH2,
    server="outlook.office365.com",
)
print(config)
current_time = time.time()
print('current time: ',current_time)
account = Account(
primary_smtp_address=user,
    config=config,
    autodiscover=False
)
print('Executed Time: ',time.time() - current_time)
print(account.root.all().count())
folder_name = getattr(account, 'inbox')
print(folder_name.all().count())
for item in folder_name.all().filter(**{'subject__icontains': 'Test2'})[:100]:
    print(item.subject)

What is the output of the print statements? Or do you get an exception?

If you enable debug output (see https://ecederstrand.github.io/exchangelib/#troubleshooting) then you can see the entire interaction between the server and the client. That should help you see what is happening in the failing case.

@ecederstrand, There is not returning any exception. Here is the output of the print statement 6d78d69a-xxxx-4410-xxxx-xxxxxx Configuration(credentials=OAuth2Credentials('6d78d69a-xxxx-4410-xxxx-xxxxxx', '********'), service_endpoint='https://outlook.office365.com/EWS/Exchange.asmx', auth_type='OAuth 2.0', version=None, retry_policy=<exchangelib.protocol.FailFast object at 0x105110450>) current time: 1610344349.677962

After printing the above output, the script is not returning any response/error, it's going in a waiting state.

After enabling the debug log output is:


DEBUG:exchangelib.protocol:Protocol __call__ cache miss. Adding key '('https://outlook.office365.com/EWS/Exchange.asmx', OAuth2Credentials('6d78d69a-xxxx-4410-xxxx-xxxxxx', '********'))'
DEBUG:requests_oauthlib.oauth2_session:Encoding `client_id` "6d78d69a-xxxx-4410-xxxx-xxxxxx" with `client_secret` as Basic auth credentials.
DEBUG:requests_oauthlib.oauth2_session:Requesting url https://login.microsoftonline.com/fedbf64c-a16b-43b0-a9e3-e2ad7fb6cf59/oauth2/v2.0/token using method POST.
DEBUG:requests_oauthlib.oauth2_session:Supplying headers {'Accept': 'application/json', 'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'} and data {'grant_type': 'client_credentials', 'scope': 'https://outlook.office365.com/.default'}
DEBUG:requests_oauthlib.oauth2_session:Passing through key word arguments {'timeout': None, 'auth': <requests.auth.HTTPBasicAuth object at 0x1017d1a10>, 'verify': True, 'proxies': None}.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): login.microsoftonline.com:443
DEBUG:urllib3.connectionpool:https://login.microsoftonline.com:443 "POST /fedbf64c-a16b-43b0-a9e3-e2ad7fb6cf59/oauth2/v2.0/token HTTP/1.1" 200 1597
DEBUG:requests_oauthlib.oauth2_session:Request to fetch token completed with status 200.
DEBUG:requests_oauthlib.oauth2_session:Request url was https://login.microsoftonline.com/fedbf64c-a16b-43b0-a9e3-e2ad7fb6cf59/oauth2/v2.0/token
DEBUG:requests_oauthlib.oauth2_session:Request headers were {'User-Agent': 'exchangelib/3.3.2 (python-requests/2.22.0)', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8', 'Content-Length': '82', 'Authorization': 'Basic base64_auth_token='}
DEBUG:requests_oauthlib.oauth2_session:Request body was grant_type=client_credentials&scope=https%3A%2F%2Foutlook.office365.com%2F.default
DEBUG:requests_oauthlib.oauth2_session:Response headers were {'Cache-Control': 'no-store, no-cache', 'Pragma': 'no-cache', 'Content-Type': 'application/json; charset=utf-8', 'Expires': '-1', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'X-Content-Type-Options': 'nosniff', 'P3P': 'CP="DSP CUR OTPi IND OTRi ONL FIN"', 'x-ms-request-id': '1a4d198d-6b40-4c96-a8b3-9018c2480000', 'x-ms-ests-server': '2.1.11384.5 - AMS1 ProdSlices', 'Set-Cookie': 'fpc=AiAe9Q2Q16NNqdNcTbqsTo6g5R_pAQAAAJTjjdcOAAAA; expires=Wed, 10-Feb-2021 06:13:41 GMT; path=/; secure; HttpOnly; SameSite=None, x-ms-gateway-slice=prod; path=/; secure; samesite=none; httponly, stsservicecookie=ests; path=/; secure; samesite=none; httponly', 'Date': 'Mon, 11 Jan 2021 06:13:41 GMT', 'Content-Length': '1597'} and content {"token_type":"Bearer","expires_in":3599,"ext_expires_in":3599,"access_token":"Base64_Access_Token"}.
DEBUG:requests_oauthlib.oauth2_session:Invoking 0 token response hooks.
DEBUG:requests_oauthlib.oauth2_session:Obtained token {'token_type': 'Bearer', 'expires_in': 3599, 'ext_expires_in': 3599, 'access_token': 'Base64_Access_Token', 'expires_at': 1610349220.65043}.
DEBUG:exchangelib.credentials:Setting auth token for 6d78d69a-xxxx-4410-xxxx-xxxxxx
DEBUG:exchangelib.protocol:Server outlook.office365.com: Created session 26570
DEBUG:exchangelib.version:Asking server for version info using API version Exchange2019
DEBUG:exchangelib.services.common:Trying API version Exchange2019
DEBUG:exchangelib.protocol:Server outlook.office365.com: Waiting for session
DEBUG:exchangelib.protocol:Server outlook.office365.com: Got session 26570
DEBUG:exchangelib.util:Session 26570 thread 4314729920: retry 0 timeout 120 POST'ing to https://outlook.office365.com/EWS/Exchange.asmx after 10s wait
DEBUG:requests_oauthlib.oauth2_session:Invoking 0 protected resource request hooks.
DEBUG:requests_oauthlib.oauth2_session:Adding token {'token_type': 'Bearer', 'expires_in': 3599, 'ext_expires_in': 3599, 'access_token': 'Base64_Access_Token', 'expires_at': 1610349220.65043} to request.
DEBUG:requests_oauthlib.oauth2_session:Requesting url https://outlook.office365.com/EWS/Exchange.asmx using method POST.
DEBUG:requests_oauthlib.oauth2_session:Supplying headers {'Authorization': 'Bearer Base64_Access_Token} and data b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><s:Header><t:RequestServerVersion Version="Exchange2019"/><t:ExchangeImpersonation><t:ConnectingSID><t:PrimarySmtpAddress>user@domain.com</t:PrimarySmtpAddress></t:ConnectingSID></t:ExchangeImpersonation></s:Header><s:Body><m:ResolveNames ReturnFullContactData="false"><m:UnresolvedEntry>6d78d69a-9e0b-4410-b16a-733b910c5cbf</m:UnresolvedEntry></m:ResolveNames></s:Body></s:Envelope>'
DEBUG:requests_oauthlib.oauth2_session:Passing through key word arguments {'json': None, 'allow_redirects': False, 'timeout': 120, 'stream': False}.
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): outlook.office365.com:443
DEBUG:urllib3.connectionpool:https://outlook.office365.com:443 "POST /EWS/Exchange.asmx HTTP/1.1" 500 753
DEBUG:exchangelib.util:Retry: 0
Waited: 10
Timeout: 120
Session: 26570
Thread: 4314729920
Auth type: <requests_oauthlib.oauth2_auth.OAuth2 object at 0x102077a10>
URL: https://outlook.office365.com/EWS/Exchange.asmx
HTTP adapter: <requests.adapters.HTTPAdapter object at 0x102596cd0>
Allow redirects: False
Streaming: False
Response time: 1.370959208
Status code: 500
Request headers: {'User-Agent': 'exchangelib/3.3.2 (python-requests/2.22.0)', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Type': 'text/xml; charset=utf-8', 'Authorization': 'Bearer Base64_Access_Token', 'Content-Length': '634'}
Response headers: {'Cache-Control': 'private', 'Content-Length': '753', 'Content-Type': 'text/xml; charset=utf-8', 'Server': 'Microsoft-IIS/10.0', 'request-id': '228e9956-67b6-456f-a833-93f77bba238f', 'X-CalculatedFETarget': 'AU2P273CU001.internal.outlook.com', 'X-BackEndHttpStatus': '500, 500', 'Set-Cookie': 'exchangecookie=6041c87c68b049e6bbf5531f65b1fb1b; expires=Tue, 11-Jan-2022 06:13:42 GMT; path=/; secure; HttpOnly', 'X-FEProxyInfo': 'AU2P273CA0022.AREP273.PROD.OUTLOOK.COM', 'X-CalculatedBETarget': 'AU2P273MB0513.AREP273.PROD.OUTLOOK.COM', 'X-RUM-Validated': '1', 'x-ms-appId': '6d78d69a-9e0b-4410-b16a-733b910c5cbf', 'X-AspNet-Version': '4.0.30319', 'X-BeSku': 'WCS6', 'X-DiagInfo': 'AU2P273MB0513', 'X-BEServer': 'AU2P273MB0513', 'X-Proxy-RoutingCorrectness': '1', 'X-Proxy-BackendServerStatus': '500', 'X-FEServer': 'AU2P273CA0022, BM1PR0101CA0059', 'X-Powered-By': 'ASP.NET', 'Date': 'Mon, 11 Jan 2021 06:13:42 GMT'}
Request data: b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><s:Header><t:RequestServerVersion Version="Exchange2019"/><t:ExchangeImpersonation><t:ConnectingSID><t:PrimarySmtpAddress>user@domain.com</t:PrimarySmtpAddress></t:ConnectingSID></t:ExchangeImpersonation></s:Header><s:Body><m:ResolveNames ReturnFullContactData="false"><m:UnresolvedEntry>6d78d69a-9e0b-4410-b16a-733b910c5cbf</m:UnresolvedEntry></m:ResolveNames></s:Body></s:Envelope>'
Response data: b'<?xml version="1.0" encoding="utf-8"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Header><Action s:mustUnderstand="1" xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none">*</Action></s:Header><s:Body><s:Fault><faultcode xmlns:a="http://schemas.microsoft.com/exchange/services/2006/types">a:ErrorInvalidServerVersion</faultcode><faultstring xml:lang="en-US">The specified server version is invalid.</faultstring><detail><e:ResponseCode xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ErrorInvalidServerVersion</e:ResponseCode><e:Message xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">The specified server version is invalid.</e:Message></detail></s:Fault></s:Body></s:Envelope>'

DEBUG:exchangelib.util:No retry: wrong status code 500
DEBUG:exchangelib.util:Got status code 500 but trying to parse content anyway
DEBUG:exchangelib.util:Session 26570 thread 4314729920: Useful response from https://outlook.office365.com/EWS/Exchange.asmx
DEBUG:exchangelib.protocol:Server outlook.office365.com: Releasing session 26570
DEBUG:exchangelib.services.common:Failed to update version info (No ServerVersionInfo in header: '<s:Header xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><Action xmlns="http://schemas.microsoft.com/ws/2005/05/addressing/none" s:mustUnderstand="1">*</Action></s:Header>')
DEBUG:exchangelib.services.common:API version Exchange2019 was invalid
DEBUG:exchangelib.services.common:Trying API version Exchange2016
DEBUG:exchangelib.protocol:Server outlook.office365.com: Waiting for session
DEBUG:exchangelib.protocol:Server outlook.office365.com: Got session 26570
DEBUG:exchangelib.util:Session 26570 thread 4314729920: retry 0 timeout 120 POST'ing to https://outlook.office365.com/EWS/Exchange.asmx after 10s wait
DEBUG:requests_oauthlib.oauth2_session:Invoking 0 protected resource request hooks.
DEBUG:requests_oauthlib.oauth2_session:Adding token {'token_type': 'Bearer', 'expires_in': 3599, 'ext_expires_in': 3599, 'access_token': 'Base64_Access_Token', 'expires_at': 1610349220.65043} to request.
DEBUG:requests_oauthlib.oauth2_session:Requesting url https://outlook.office365.com/EWS/Exchange.asmx using method POST.
DEBUG:requests_oauthlib.oauth2_session:Supplying headers {'Authorization': 'Bearer Base64_Access_Token'} and data b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><s:Header><t:RequestServerVersion Version="Exchange2016"/><t:ExchangeImpersonation><t:ConnectingSID><t:PrimarySmtpAddress>user@domain.com</t:PrimarySmtpAddress></t:ConnectingSID></t:ExchangeImpersonation></s:Header><s:Body><m:ResolveNames ReturnFullContactData="false"><m:UnresolvedEntry>6d78d69a-9e0b-4410-b16a-733b910c5cbf</m:UnresolvedEntry></m:ResolveNames></s:Body></s:Envelope>'
DEBUG:requests_oauthlib.oauth2_session:Passing through key word arguments {'json': None, 'allow_redirects': False, 'timeout': 120, 'stream': False}.
DEBUG:urllib3.connectionpool:https://outlook.office365.com:443 "POST /EWS/Exchange.asmx HTTP/1.1" 500 806
DEBUG:exchangelib.util:Retry: 0
Waited: 10
Timeout: 120
Session: 26570
Thread: 4314729920
Auth type: <requests_oauthlib.oauth2_auth.OAuth2 object at 0x102077a10>
URL: https://outlook.office365.com/EWS/Exchange.asmx
HTTP adapter: <requests.adapters.HTTPAdapter object at 0x102596cd0>
Allow redirects: False
Streaming: False
Response time: 0.2104482239999994
Status code: 500
Request headers: {'User-Agent': 'exchangelib/3.3.2 (python-requests/2.22.0)', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Type': 'text/xml; charset=utf-8', 'Authorization': 'Bearer Base64_Access_Token', 'Cookie': 'exchangecookie=6041c87c68b049e6bbf5531f65b1fb1b', 'Content-Length': '634'}
Response headers: {'Cache-Control': 'private', 'Content-Length': '806', 'Content-Type': 'text/xml; charset=utf-8', 'Server': 'Microsoft-IIS/10.0', 'request-id': 'f3f29530-2adc-4a91-a655-f7317a22225b', 'X-CalculatedFETarget': 'DXXP273CU001.internal.outlook.com', 'X-BackEndHttpStatus': '500, 500', 'Set-Cookie': 'exchangecookie=6041c87c68b049e6bbf5531f65b1fb1b; path=/; secure', 'X-FEProxyInfo': 'DXXP273CA0008.AREP273.PROD.OUTLOOK.COM', 'X-CalculatedBETarget': 'DX2P273MB0026.AREP273.PROD.OUTLOOK.COM', 'X-RUM-Validated': '1', 'x-ms-appId': '6d78d69a-9e0b-4410-b16a-733b910c5cbf', 'X-AspNet-Version': '4.0.30319', 'X-BeSku': 'WCS5', 'X-DiagInfo': 'DX2P273MB0026', 'X-BEServer': 'DX2P273MB0026', 'X-Proxy-RoutingCorrectness': '1', 'X-Proxy-BackendServerStatus': '500', 'X-FEServer': 'DXXP273CA0008, BM1PR0101CA0059', 'X-Powered-By': 'ASP.NET', 'Date': 'Mon, 11 Jan 2021 06:13:42 GMT'}
Request data: b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\n<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><s:Header><t:RequestServerVersion Version="Exchange2016"/><t:ExchangeImpersonation><t:ConnectingSID><t:PrimarySmtpAddress>user@domain.com</t:PrimarySmtpAddress></t:ConnectingSID></t:ExchangeImpersonation></s:Header><s:Body><m:ResolveNames ReturnFullContactData="false"><m:UnresolvedEntry>6d78d69a-9e0b-4410-b16a-733b910c5cbf</m:UnresolvedEntry></m:ResolveNames></s:Body></s:Envelope>'
Response data: b'<?xml version="1.0" encoding="utf-8"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><s:Fault><faultcode xmlns:a="http://schemas.microsoft.com/exchange/services/2006/types">a:ErrorNonExistentMailbox</faultcode><faultstring xml:lang="en-US">The SMTP address has no mailbox associated with it.</faultstring><detail><e:ResponseCode xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ErrorNonExistentMailbox</e:ResponseCode><e:Message xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">The SMTP address has no mailbox associated with it.</e:Message><t:MessageXml xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"><t:Value Name="SmtpAddress">user@domain.com</t:Value></t:MessageXml></detail></s:Fault></s:Body></s:Envelope>'

DEBUG:exchangelib.util:No retry: wrong status code 500
DEBUG:exchangelib.util:Got status code 500 but trying to parse content anyway
DEBUG:exchangelib.util:Session 26570 thread 4314729920: Useful response from https://outlook.office365.com/EWS/Exchange.asmx
DEBUG:exchangelib.protocol:Server outlook.office365.com: Releasing session 26570
DEBUG:exchangelib.services.common:No header in XML response

We expect a "The SMTP address has no mailbox associated with it" exception or error message.

Ok, in this case it seems we go into an infinite loop trying to guess the server version when the SMTP address is wrong. That's a bug. I'll try to recreate this on my test server and find a solution.

@ecederstrand I'm trying to connect the exchange mailbox using exchangelib for OAuth2 on Microsoft Exchange, I have added full_app permission for the app. first API call works as expected and print subscription ID. but the second API call returns a 500 internal server error with ExchangeImpersonation SOAP header must be present for this type of OAuth token. I have not much familiar with this error. any help would be appreciated

python sample script as below:


from exchangelib import Account, Configuration, Credentials, OAuth2Credentials, Identity, OAUTH2
from urllib.parse import urlparse
from exchangelib.protocol import BaseProtocol, NoVerifyHTTPAdapter
import requests_oauthlib

client_id='client_id'
client_secret='client_secret'
tenant_id='tenant_id'
username='email_id'
host='outlook.office365.com'
auth_type='Modern Auth'
email='email_id'
access_type='impersonation'
verify_ssl=False
auth_method='Modern Auth'
password='*************'

def get_credentials(auth_type, username, password, email, client_id, client_secret, tenant_id):
    if auth_type == OAUTH2:
        credentials = OAuth2Credentials(
            client_id=client_id, client_secret=client_secret, tenant_id=tenant_id, identity=Identity(primary_smtp_address=email)
        )
    else:
        credentials = Credentials(username=username, password=password)
    return credentials

def get_exchange_client(email, credentials, server, access_type, auth_type, use_autodiscover=False, verify_ssl=False):
    try:
        access_type = access_type if access_type else 'delegate'
        parse_object = urlparse(server)
        if parse_object.scheme:
            server = parse_object.netloc
        if not verify_ssl:
            BaseProtocol.HTTP_ADAPTER_CLS = NoVerifyHTTPAdapter
        if use_autodiscover:
            return Account(primary_smtp_address=email, credentials=credentials,
                           autodiscover=use_autodiscover, access_type=access_type.lower())
        else:
            config = Configuration(server=server, credentials=credentials, auth_type=auth_type)
            return Account(primary_smtp_address=email, config=config, access_type=access_type.lower())
    except Exception as err:
        print(err)

auth_type = OAUTH2 if auth_method == 'Modern Auth' else None
credentials = get_credentials(auth_type, username, password, email, client_id, client_secret, tenant_id)
client = get_exchange_client(email, credentials, host, access_type, auth_type, verify_ssl=verify_ssl)
print(client)
print(client.protocol.credentials.access_token)

import requests, xmltodict

service_url='https://outlook.office365.com/EWS/Exchange.asmx'

mailbox='user@domain.onmicrosoft.com'

request_body="""<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>
    <t:RequestServerVersion Version="Exchange2016" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" soap:mustUnderstand="0" />
    <t:ExchangeImpersonation>
      <t:ConnectingSID>
        <t:SmtpAddress>user@domain.onmicrosoft.com</t:SmtpAddress>
      </t:ConnectingSID>
    </t:ExchangeImpersonation>
  </soap:Header>
  <soap:Body>
    <m:Subscribe>
      <m:StreamingSubscriptionRequest>
        <t:FolderIds>
          <t:DistinguishedFolderId Id="inbox" />
        </t:FolderIds>
        <t:EventTypes>
          <t:EventType>NewMailEvent</t:EventType>
          <t:EventType>ModifiedEvent</t:EventType>
          <t:EventType>MovedEvent</t:EventType>
        </t:EventTypes>
      </m:StreamingSubscriptionRequest>
    </m:Subscribe>
  </soap:Body>
</soap:Envelope>"""

headers = {'content-type': 'text/xml'}

response = requests.post(service_url, auth=requests_oauthlib.OAuth2(client_id=client_id,token=client.protocol.credentials.access_token), data=request_body, headers=headers, verify=False, timeout=90)

if response.ok:
    try:
        json_response = xmltodict.parse(response.text)
        id = json_response['s:Envelope']['s:Body']['m:SubscribeResponse']['m:ResponseMessages'][
            'm:SubscribeResponseMessage']['m:SubscriptionId']
        if id:
            print("Notification service successfully subscribe")
            print("subscription id :{}".format(id))

        body="""<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages"
               xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"
               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>
    <t:RequestServerVersion Version="Exchange2016" xmlns="http://schemas.microsoft.com/exchange/services/2006/types" soap:mustUnderstand="0" />
  </soap:Header>
  <soap:Body>
    <m:GetStreamingEvents>
      <m:SubscriptionIds>
        <t:SubscriptionId>{id}</t:SubscriptionId>
      </m:SubscriptionIds>
      <m:ConnectionTimeout>1</m:ConnectionTimeout>
    </m:GetStreamingEvents>
  </soap:Body>
</soap:Envelope>""".format(id=id)

        response2 = requests.post(service_url, auth=requests_oauthlib.OAuth2(client_id=client_id, token=client.protocol.credentials.access_token), data=body,
                                 headers=headers, verify=False, timeout=90)
        if response2.ok:
            print('Response2 --->',response2.text)

        else:
            print("Error in unsubscribe notification service")
    except Exception as e:
        print(e)
else:
    print("Error in subscribe notification service")

Response for a script as below:

 InsecureRequestWarning,
EXCHANGE CLIENT:  user@domain.onmicrosoft.com
{'token_type': 'Bearer', 'expires_in': 3599, 'ext_expires_in': 3599, 'access_token': '********************', 'expires_at': 1644941194.524418}

Notification service successfully subscribe

subscription id : '****************'

Error in unsubscribe notification service--->>>> <?xml version="1.0" encoding="utf-8"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body><s:Fault><faultcode xmlns:a="http://schemas.microsoft.com/exchange/services/2006/types">a:ErrorInvalidExchangeImpersonationHeaderData</faultcode><faultstring xml:lang="en-US">ExchangeImpersonation SOAP header must be present for this type of OAuth token.</faultstring><detail><e:ResponseCode xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ErrorInvalidExchangeImpersonationHeaderData</e:ResponseCode><e:Message xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ExchangeImpersonation SOAP header must be present for this type of OAuth token.</e:Message></detail></s:Fault></s:Body></s:Envelope>

Ok. I think this happens during version guessing. The last stack trace you posted doesn't contain enough steps to confirm this. Can you post a full stack trace - a trace that starts in your own code?

If this is indeed raised during version guessing, it's a bug in exchangelib. It's not going to be easy to fix. The root problem is that some services, e.g. ResolveNames, don't require account information, but account information is needed to construct the impersonation SOAP headers.

Anyway, I think you can work around this by supplying a version argument for your Configuration object. Something like this should do for O365:
from exchangelib.version import Version, EXCHANGE_O365

config = Configuration(
    ...,
    version=Version(build=EXCHANGE_O365),
)
All services accessible via account.protocol will still be broken when you are using this specific OAuth authentication mechanism, until this bug is fixed.

The above gives me the following error.

Stack trace:

Traceback (most recent call last): File "ouathTest.py", line 25, in for item in account.inbox.all(): File "site-packages\cached_property.py", line 74, in get return obj_dict.setdefault(name, self.func(obj)) File "site-packages\exchangelib\account.py", line 271, in inbox return self.root.get_default_folder(Inbox) File "site-packages\cached_property.py", line 74, in get return obj_dict.setdefault(name, self.func(obj)) File "site-packages\exchangelib\account.py", line 335, in root return Root.get_distinguished(account=self) File "exchangelib\folders\roots.py", line 113, in get_distinguished return cls.resolve(account=account, folder=DistinguishedFolderId(id=cls.DISTINGUISHED_FOLDER_ID)) File "site-packages\exchangelib\folders\base.py", line 510, in resolve folders = list(FolderCollection(account=account, folders=[folder]).resolve()) File "site-packages\exchangelib\folders\collections.py", line 335, in resolve yield from self.class(account=self.account, folders=resolveable_folders).get_folders( File "site-packages\exchangelib\folders\collections.py", line 403, in get_folders yield from GetFolder(account=self.account).call( File "site-packages\exchangelib\services\get_folder.py", line 43, in _elems_to_objs for folder, elem in zip(self.folders, elems): File "site-packages\exchangelib\services\common.py", line 287, in _chunked_get_elements yield from self._get_elements(payload=payload_func(chunk, *kwargs)) File "site-packages\exchangelib\services\common.py", line 308, in _get_elements yield from self._response_generator(payload=payload) File "site-packages\exchangelib\services\common.py", line 271, in _response_generator response = self._get_response_xml(payload=payload) File "site-packages\exchangelib\services\common.py", line 404, in _get_response_xml r = self._get_response(payload=payload, api_version=api_version) File "site-packages\exchangelib\services\common.py", line 355, in _get_response r, session = post_ratelimited( File "site-packages\exchangelib\util.py", line 874, in post_ratelimited protocol.retry_policy.raise_response_errors(r) File "exchangelib\protocol.py", line 721, in raise_response_errors raise MalformedResponseError( exchangelib.errors.MalformedResponseError: Unknown failure in response. Code: 403 headers: {'Cache-Control': 'private', 'Content-Type': 'text/xml; charset=utf-8', 'Server': 'Microsoft-IIS/10.0',
'request-id': 'f157b5ae-f508-dcce-c9db-83177d2f12b3', 'Alt-Svc': 'h3=":443",h3-29=":443"', 'X-CalculatedFETarget': 'PA7P264CU030.internal.outlook.com', 'X-BackEndHttpStatus': '403, 403',
'Set-Cookie': 'exchangecookie=a08b3d6460c043f9a6a4812cc8349f1e; expires=Sat, 14-Dec-2024 18:07:37 GMT; path=/; secure; HttpOnly', 'WWW-Authenticate': 'Bearer client_id="00000002-0000-0ff1-ce00-000000000000", trusted_issuers="00000001-0000-0000-c000-000000000000@", token_types="app_asserted_user_v1 service_asserted_app_v1", error="invalid_token"', 'X-CalculatedBETarget': 'PAVPR02MB9556.eurprd02.prod.outlook.com', 'X-RUM-Validated': '1', 'X-RUM-NotUpdateQueriedPath': '1', 'X-RUM-NotUpdateQueriedDbCopy': '1', 'x-ms-appId': 'f40c7bac-4dcb-447a-9934-fe7a97fa6631', 'Restrict-Access-Confirm': '1', 'x-ms-diagnostics': '2000008;reason="The token contains not enough scope to make this call.";error_category="invalid_grant"', 'X-AspNet-Version': '4.0.30319', 'X-BeSku': 'WCS7', 'X-DiagInfo': 'PAVPR02MB9556', 'X-BEServer': 'PAVPR02MB9556', 'X-Proxy-RoutingCorrectness': '1', 'X-Proxy-BackendServerStatus': '403', 'X-FEProxyInfo': 'LO2P265CA0130.GBRP265.PROD.OUTLOOK.COM', 'X-FEEFZInfo': 'LHR', 'X-FEServer': 'PA7P264CA0536, LO2P265CA0130', 'X-FirstHopCafeEFZ': 'LHR', 'X-Powered-By': 'ASP.NET', 'Date': 'Thu, 14 Dec 2023 18:07:37 GMT', 'Content-Length': '0'} content:

That response from the server contains the actual issue: "The token contains not enough scope to make this call.". You need to create an OAuth token with sufficient permissions to use EWS.

ecederstrand / exchangelib

OAuth2Credentials requires SOAP impersonation headers unconditionally #735