ConnectionError: RemoteDisconnected - Remote end closed connection without response

[anyu]

• [X] I have checked that the SDK documentation doesn't solve my issue.
• [X] I have checked that the API documentation doesn't solve my issue.
• [X] I have searched the Box Developer Forums and my issue isn't already reported (or if it has been reported, I have attached a link to it, for reference).
• [X] I have searched Issues in this repo and my issue isn't already reported.

Description of the Issue

Our app sends a consistent volume of requests at regular intervals to the GET folder information endpoint.

Over the last 2 months at least, we've started seeing that a handful fail in between successful requests due to the RemoteDisconnected error below. The only notable pattern identified so far has been that there seems to be an uptick in these errors around mid-month through the end of the month, despite consistent request volume.

If we're hitting any sort of rate limit, we'd expect a 429, which we do get for other requests.

Are there any additional limits from Box's end that would result in such errors, and not be 429s?

The docs on Rate Limits has a section that says

If Box's infrastructure detects that a tool used by or on behalf of a customer has exceeded that customer's API license allocation or is intending to circumvent network controls, additional selective rate-limiting may be imposed.

Would these 'additional selective rate-limiting measures' return 429s?

Error Message, Including Stack Trace

["\u003cclass 'requests.exceptions.ConnectionError'\u003e","('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))","\u003ctraceback object at 0x7f74d558e300\u003e"]

Versions Used

Python SDK: boxsdk 3.3.0 Python: 3.9.13

[mwwoda]

Hi @anyu It looks like your connection was forcibly closed before receiving a response. This could be a server issue, or it could happen if your environment has changed between requests (such as network issues).

I don't think we can easily reproduce this issue in sdk because of its non-deterministic nature and it looks more like a server problem than a sdk problem. My advice is to contact Box Support https://support.box.com/hc/en-us/requests/new so that this problem can be further investigated on the backend side.

[anyu]

We've chatted with Box Support and unfortunately a cause hasn't been identified.

[anyu]

We've chatted with Box Support and unfortunately a cause hasn't been identified.

For posterity, we ended up creating a new Box client for each of our job processes making these calls (instead of globally caching them), and that got around the issue.

[lukaszsocha2]

I guess with the fix released in pythonsdk 3.5.1 it won't be necessary any more. You should be able to cache and reuse one client object now.

[matthewpick]

@anyu I also faced some odd 15min delayed request followed by a RemoteDisconnected warning. This Github issue is what allowed me to trace the problem to some caching I was doing on the BoxClient.

Warning:  [2024-01-24 03:25:18.419Z] [default_network::_log_exception:93] Request "POST https://upload.box.com/api/2.0/files/content" failed with ConnectionError exception: ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))
Warning:  [2024-01-24 03:41:00.499Z] [default_network::_log_exception:93] Request "GET https://api.box.com/2.0/files/1234578910" failed with ConnectionError exception: ConnectionError(ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))

Thank you for documenting your finding! I was seeing very intermittent issues with box api requests, the only way I could reproduce this issue was by running my box data pipeline e2e test suite multiple times in a row.

I removed the functools @cache I was using on my BoxClient session and that seems to have resolved my issue.

For additional context: I was facing BoxClient caching issues with version 3.7.3 of the boxsdk.