vgrem / Office365-REST-Python-Client

Microsoft 365 & Microsoft Graph Library for Python
MIT License
1.27k stars 331 forks source link

SSLCertVerificationError - AttributeError: 'NoneType' object has no attribute 'text' #661

Open Barnard-87 opened 1 year ago

Barnard-87 commented 1 year ago

Code worked completely fine and out of nowhere started giving me this error message: AttributeError: 'NoneType' object has no attribute 'text'

Only other issues I can find online are for the Sharepy libary here on Github and here on Stack

Not sure what else can be tested on my end or what the Systems Admin can do to resolve this issue. If I could better target where to troubleshoot (login credentials, code to access, etc) that would be optimal.

The Systems Admin and I have tried using new credentials with no success, and I have also tried uninstalling and reinstalling the library, and testing on a new Python Environment, to no success.

Code used in script:

from office365.sharepoint.client_context import ClientContext
from office365.runtime.auth.client_credential import ClientCredential

client_credentials = ClientCredential(client_id,client_secret)
ctx = ClientContext(sitepath).with_credentials(client_credentials)

web = ctx.web
ctx.load(web)
ctx.execute_query()

The last line is where the error is thrown when I run it line by line.

Full error code:

SSLCertVerificationError                  Traceback (most recent call last)
[c:\Users\restoffilepath\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    702             # Make the request on the httplib connection object.
--> 703             httplib_response = self._make_request(
    704                 conn,

[c:\Users\restoffilepath\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    385         try:
--> 386             self._validate_conn(conn)
    387         except (SocketTimeout, BaseSSLError) as e:

[c:\Users\restoffilepath\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn)
   1039         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
-> 1040             conn.connect()
   1041 

[c:\Users\restoffilepath\lib\site-packages\urllib3\connection.py in connect(self)
    415 
--> 416         self.sock = ssl_wrap_socket(
    417             sock=conn,

[c:\Users\restoffilepath\lib\site-packages\urllib3\util\ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data, tls_in_tls)
    448     if send_sni:
--> 449         ssl_sock = _ssl_wrap_socket_impl(
...
---> 45             self.error = e.response.text
     46             raise ValueError(e.response.text)
     47 

AttributeError: 'NoneType' object has no attribute 'text'
eu2pey4 commented 1 year ago

My experience is exactly the same. Working completely fine for months before 3/14/2023, suddenly this exact same error. Occuring across multiple OSs such as Windows 10 and Windows Server 2016.

Code used in script:

from office365.runtime.auth.client_credential import ClientCredential
from office365.sharepoint.client_context import ClientContext
from office365.sharepoint.sharing.role_type import RoleType

sharepoint_url = f'https://{self.domain}/sites/{self.siteName}'

# Initialize the client credentials
client_credentials = ClientCredential(client_id, client_secret)

# create client context object
self.client_context = ClientContext(sharepoint_url).with_credentials(client_credentials)

self.web = self.client_context.web.get().execute_query()

Stack Trace:

Traceback (most recent call last):
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connection.py", line 424, in connect
    tls_in_tls=tls_in_tls,
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\util\ssl_.py", line 450, in ssl_wrap_socket
    sock, context, tls_in_tls, server_hostname=server_hostname
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\ssl.py", line 423, in wrap_socket
    session=session
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\ssl.py", line 870, in _create
    self.do_handshake()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\adapters.py", line 499, in send
    timeout=timeout,
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 788, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\util\retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='accounts.accesscontrol.windows.net', port=443): Max retries exceeded with url: /da67ef1b-ca59-4db2-9a8c-aa8d94617a16/tokens/OAuth/2 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 43, in get_app_only_access_token
    return self._get_app_only_access_token(url_info.hostname, realm)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 65, in _get_app_only_access_token
    data=oauth2_request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='accounts.accesscontrol.windows.net', port=443): Max retries exceeded with url: /da67ef1b-ca59-4db2-9a8c-aa8d94617a16/tokens/OAuth/2 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:/Users/eu2pey4/OneDrive - McKesson Corporation/Source Code/Muse/Automation/MailReports/GxCostReport.py", line 51, in <module>
    report = GxCostReport(user='conner.holt@mckesson.com')
  File "c:/Users/eu2pey4/OneDrive - McKesson Corporation/Source Code/Muse/Automation/MailReports/GxCostReport.py", line 16, in __init__
    self.sharepointApi = SharePointApi(siteName='SpecialtyPricing')
  File "C:\Users\eu2pey4\OneDrive - McKesson Corporation\Source Code\Muse\APIs\SharepointApi.py", line 45, in __init__
    self.web = self.client_context.web.get().execute_query()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_object.py", line 44, in execute_query
    self.context.execute_query()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_runtime_context.py", line 162, in execute_query
    self.pending_request().execute_query(qry)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_request.py", line 57, in execute_query
    response = self.execute_request_direct(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_request.py", line 69, in execute_request_direct
    self.beforeExecute.notify(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\types\event_handler.py", line 21, in notify
    listener(*args, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\sharepoint\client_context.py", line 212, in _authenticate_request
    self.authentication_context.authenticate_request(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\authentication_context.py", line 97, in authenticate_request
    self._provider.authenticate_request(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 31, in authenticate_request
    self.ensure_app_only_access_token()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 36, in ensure_app_only_access_token
    self._cached_token = self.get_app_only_access_token()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 45, in get_app_only_access_token
    self.error = e.response.text
AttributeError: 'NoneType' object has no attribute 'text'

(Muse) C:\Users\eu2pey4\OneDrive - McKesson Corporation\Source Code\Muse\Automation>C:/Users/eu2pey4/Anaconda3/envs/Muse/python.exe "c:/Users/eu2pey4/OneDrive - McKesson Corporation/Source 
Code/Muse/Automation/MailReports/GxCostReport.py"
Initiating login request with your identity provider. A browser window should have opened for you to complete the login. If you can't see it, check existing browser windows, or your OS settings. Press CTRL+C to abort and try again...
Traceback (most recent call last):
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 710, in urlopen
    chunked=chunked,
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connection.py", line 424, in connect
    tls_in_tls=tls_in_tls,
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\util\ssl_.py", line 450, in ssl_wrap_socket
    sock, context, tls_in_tls, server_hostname=server_hostname
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\ssl.py", line 423, in wrap_socket
    session=session
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\ssl.py", line 870, in _create
    self.do_handshake()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\ssl.py", line 1139, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\adapters.py", line 499, in send
    timeout=timeout,
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\connectionpool.py", line 788, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\urllib3\util\retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='accounts.accesscontrol.windows.net', port=443): Max retries exceeded with url: /da67ef1b-ca59-4db2-9a8c-aa8d94617a16/tokens/OAuth/2 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 43, in get_app_only_access_token
    return self._get_app_only_access_token(url_info.hostname, realm)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 65, in _get_app_only_access_token
    data=oauth2_request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\requests\adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='accounts.accesscontrol.windows.net', port=443): Max retries exceeded with url: /da67ef1b-ca59-4db2-9a8c-aa8d94617a16/tokens/OAuth/2 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:/Users/eu2pey4/OneDrive - McKesson Corporation/Source Code/Muse/Automation/MailReports/GxCostReport.py", line 51, in <module>
    report = GxCostReport(user='conner.holt@mckesson.com')
  File "c:/Users/eu2pey4/OneDrive - McKesson Corporation/Source Code/Muse/Automation/MailReports/GxCostReport.py", line 16, in __init__
    self.sharepointApi = SharePointApi(siteName='SpecialtyPricing')
  File "C:\Users\eu2pey4\OneDrive - McKesson Corporation\Source Code\Muse\APIs\SharepointApi.py", line 43, in __init__
    self.web = self.client_context.web.get().execute_query()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_object.py", line 44, in execute_query
    self.context.execute_query()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_runtime_context.py", line 162, in execute_query
    self.pending_request().execute_query(qry)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_request.py", line 57, in execute_query
    response = self.execute_request_direct(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\client_request.py", line 69, in execute_request_direct
    self.beforeExecute.notify(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\types\event_handler.py", line 21, in notify
    listener(*args, **kwargs)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\sharepoint\client_context.py", line 212, in _authenticate_request
    self.authentication_context.authenticate_request(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\authentication_context.py", line 97, in authenticate_request
    self._provider.authenticate_request(request)
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 31, in authenticate_request
    self.ensure_app_only_access_token()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 36, in ensure_app_only_access_token
    self._cached_token = self.get_app_only_access_token()
  File "C:\Users\eu2pey4\Anaconda3\envs\Muse\lib\site-packages\office365\runtime\auth\providers\acs_token_provider.py", line 45, in get_app_only_access_token
    self.error = e.response.text
AttributeError: 'NoneType' object has no attribute 'text'
Barnard-87 commented 1 year ago

I'm glad this came out of nowhere for someone else as well. I thought something may have changed on my organization's side (which is still out of my control - but at least might have been fixable). Now it appears this is (most likely) a known issue that is 100% out of our control.
I'd like to avoid creating a Power Automate Flow to get the content I need downloaded in .CSVs and attachment download folders, but depending on how long this goes unresolved I might need to work on that workaround.

Barnard-87 commented 1 year ago

@eu2pey4 Can I ask how frequently this script was run? Mine began to stop working before 3/14, but only by a few more days. Not sure if your script ran daily or the 14th just happened to be the first time it ran since it stopped.
My hypothesis is that Microsoft is rolling out some sort of update to SP that results in these errors being thrown out.

eu2pey4 commented 1 year ago

@Barnard-87 My script runs weekly, so the issue likely began when yours stopped working, not mine.

Barnard-87 commented 1 year ago

@vgrem Do you have any knowledge if this is being looked into? I'm debating alternate workflows to utilize Power Automate if this issue isn't going to be resolved anytime soon but don't want to waste my time if this is being looked into or fixed. Cheers.

vgrem commented 1 year ago

Greetings @Barnard-87,

assuming app principal is registered via Azure AD, perhaps:

Barnard-87 commented 1 year ago

@vgrem thanks for the response! I'll have to double check nothing changed with Active Directory but that should be all set, and we've created new secret key and certificate so that is ruled out. To verify permissions, is that something done within Python? Apologies for lack of knowledge there. Cheers.

vgrem commented 1 year ago

@Barnard-87, in terms of permissions, how to grant permissions per application could be found in this article , section API permissions.

In this approach, permissions are granted on tenant level, across all site collections, alternately permissions could be granted per specific site collection as described here

eu2pey4 commented 1 year ago

@Barnard-87 Did vgrem's information resolve your issue? Wondering why permissions would suddenly become an issue when they were not for so long.

Barnard-87 commented 1 year ago

@eu2pey4 Hey again, unfortunately my Systems Admin went on vacation starting yesterday and hoped to take a look before they left but didn't get the chance. I don't have the permissions to check on those but I'm either going to be finding another Admin or starting on working on alternative workflows while they're out. Hoping for the best here, this roadblock completely halted a project I was working on.

Barnard-87 commented 1 year ago

@vgrem @eu2pey4 All possible troubleshooting has been done between Registering as a new App Principal and given all Permissions - to which the same error message is occuring.

As of now we're deeming this unusable until further notice and will be moving forward with another workflow to get the SP tables and attachments into Python, probably some use of Power Automate.

I'm happy to provide any other information that might help any developers solve this issue, as I see many implementations in the future using this library for reading our SP data. I'll be following this Issue closely to hopefully track any updates.

eu2pey4 commented 1 year ago

@Barnard-87 I deployed my automation to Airflow, which is on Linux, and it is working there! Still not working on Windows. So the bug may be isolated to Windows. That could be a workaround for you, to use Docker on your Windows machine or Linux to be able to keep running your automation as is.

aayushkatariaa commented 1 year ago

Any updates on this? This issue is still happening but in a different location now for me.

Barnard-87 commented 1 year ago

@aayushkatariaa Just gave me old script a run again - I don't see the same error I was getting before, but a different error that I think has to do with my code. So, in short, I believe the issue went away / was resolved for now? @vgrem Can you speak at all if this was fixed or just happens to be working on my end?
For now, I'm going to avoid this library as I do not want to encounter this error again after developing around it further, but if this was officially fixed I'll consider using it. Cheers.

aayushkatariaa commented 1 year ago

@Barnard-87 hey there, since I've been looking into developing a similar solution as you, could you lmk about what module you've ended up finally using/how it has turned out? Thanks.

ekoutsoff commented 8 months ago

I get the same exact error. Any updates?

aayushkatariaa commented 8 months ago

I ended up switching over to copies of the data i had to use being referenced via OneDrive. There's no good/simple solution for this exact thing and so i ended up dropping the thing altogether after spending almost an entire month over this.

Barnard-87 commented 8 months ago

Hello @aayushkatariaa @ekoutsoff I actually ended up putting the entire project on backburner as this issue has no fix in sight. I've worked with my entire company IT and troubleshooted every issue we could have.
My plan is to use FME (Feature Manipulation Engine) by Safe Software to read the data from SharePoint. Just waiting on the license to do that.

mxblsdl commented 2 months ago

I recently ran into this issue trying to run a script with python 3.10. The exact same script runs as expected with python 3.11. Wondering if other people have noticed python version differences.

ekoutsoff commented 1 month ago

@Barnard-87 Barnard-87 Just an update. Although I had the exact same error, it was due to certificate issue (Zscaler) which I fixed. So, currenty I'm back using the Office365-REST-Python-Client with no errors. Thank you!

mxblsdl commented 1 month ago

The answer here fixed this issue for me. It was ultimately an issue with SSL certificates https://stackoverflow.com/a/65860355/9362018

pip install pip-system-certs