search

box / box-python-sdk

Box SDK for Python
http://opensource.box.com/box-python-sdk/
Apache License 2.0
418 stars 215 forks source link

JWT refresh does not work #765

Closed SakethValiveti closed 1 year ago

SakethValiveti commented 1 year ago

Description of the Issue

To put it simply, the Box Python SDK is not auto refreshing the access token.

I noticed this issue while performing the manual chunk upload.

Similar issue: https://github.com/box/box-windows-sdk-v2/issues/713

Steps to Reproduce

  1. Create JWT credentials for custom app inside Box and store in local system
  2. Create python3 file which does only one single task - described in step 3
  3. Go to https://github.com/box/box-python-sdk/blob/main/docs/usage/files.md#manual-process
  4. Copy the code under there and paste in python3 code file created in step 2.
  5. Load the JWT creds from step 1 into a variable in the python code and use the following code to create an authenticated Box client: config = JWTAuth.from_settings_dictionary(jwtCredHolder) #jwtCredHolder is where the JWT is loaded into

    Create the authenticated client

    box_client = Client(config)

  6. Using dd if=/dev/urandom of=temp_40GB_file bs=1 count=0 seek=40g (MacOS) create a large 40 GB file
  7. Change references in the copied code such that it points to the file generated in step 5
  8. Execute the code
  9. In just over an hour, the error described should occur.

Expected Behavior

Expected behavior is SDK handles refreshing of access token every 50min during chunk upload of large file to Box.

Error Message, Including Stack Trace

Below is stack trace, the first line is just a print of datetime (in EST) of when the stream upload was initiated. 2022-10-13 14:19:31.948761 "PUT https://upload.box.com/api/2.0/files/upload_sessions/5A0DCFA7522B3F658800E3510CD060D0" 401 0 {'Server': 'nginx', 'Date': 'Thu, 13 Oct 2022 19:23:43 GMT', 'Content-Type': 'application/octet-stream; charset=UTF-8', 'Content-Length': '0', 'X-Envoy-Upstream-Service-Time': '225', 'WWW-Authenticate': 'Bearer realm="Service", error="invalid_token", error_description="The access token provided is invalid."', 'X-Box-Original-Ingress-ADC-Host': 'prod-c1-c-traffic-manager-4jsh', 'Strict-Transport-Security': 'max-age=31536000', 'Via': '1.1 google', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Connection': 'close'} b''

Request "POST https://api.box.com/oauth2/token" failed with ConnectionError exception: ConnectionError(ProtocolError('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1377, in getresponse response.begin() File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 320, in begin version, status, reason = self._read_status() File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 281, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1242, in recv_into return self.read(nbytes, buffer) File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1100, in read return self._sslobj.read(len, buffer) ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1377, in getresponse response.begin() File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 320, in begin version, status, reason = self._read_status() File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 281, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1242, in recv_into return self.read(nbytes, buffer) File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1100, in read return self._sslobj.read(len, buffer) urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

Versions Used

Python SDK: boxsdk-3.5.0 Python: Python 3.9.13

lukaszsocha2 commented 1 year ago

Hi @SakethValiveti, thanks for posting this issue. We are now actively investigating this issue. Python SDK refreshes token once it is expired. In your stack trace we can see that after getting 401 (token has expired), it tries to get a new token with POST https://api.box.com/oauth2/token, however it fails. I tried to reproduce this on my side and after ~70 minutes token expired, but SDK managed to get a new token and upload continued. We suspect that version of used OpenSSL may have influence on whether issue occurs or not.

  1. Can you tell us which version of cryptography are you using (pip show cryptography)?
  2. Can you also check which OpenSSL version your cryptography library uses - please paste output of this snippet: 
    from cryptography.hazmat.backends import default_backend
print(default_backend())

    @lukaszsocha2

SakethValiveti commented 1 year ago

Hi @lukaszsocha2 Here's the info you requested:

  1. cryptography version : 36.0.0
  2. <OpenSSLBackend(version: OpenSSL 1.1.1l 24 Aug 2021, FIPS: False)>
lukaszsocha2 commented 1 year ago

Hi @SakethValiveti, thanks for that. Can you also show me OpenSSL used by your system (openssl version) and version of requests library (pip show requests). Thanks

SakethValiveti commented 1 year ago

@lukaszsocha2 here's the info: openssl version : LibreSSL 2.8.3 requests : 2.25.1

Please note the work-around I found is to re-instantiate the client for every part. Please let me know if every one of these calls counts to monthly API quota.

lukaszsocha2 commented 1 year ago

Hi @SakethValiveti, can you please test if this pr fix your issue - could you checkout branch sdk-2732-fix-connection-reset-error and use code from this branch to execute the code like before. Then please tell me if the issue still occurs? Please don't change default socket options for this test as you did above.

So I will be very grateful if you could verify if this fix helps. Unfortunately this error is so indeterministic and I wasn't able to reproduce it on my own so I cannot be sure that this error will be gone now. Best, @lukaszsocha2