Server Response Errror (Bad Request) when overwriting large hyperfile since v0.26

[kykrueger]

Describe the bug When updating from version 0.25 to 0.26, we found that one of our scripts which overwrites a datasource started failing. We have narrowed down the source of the bug to the following change: https://github.com/tableau/server-client-python/commit/307d8a20a30f32c1ce615cca7c6a78b9b9bff081#r130310838

This error occurs with our servers running version 2022.1.13,

Versions Details of your environment, including:

• Tableau Server version: bug present when using 2022.1.13, but not with 2022.1.16
• Python version 3.9
• TSC library version 0.26+

To Reproduce Upload a large hyperfile, ours causing the error is 150+ MB.

Results

    raise ServerResponseError.from_response(server_response.content, self.parent_srv.namespace, url)
tableauserverclient.server.endpoint.exceptions.ServerResponseError:

        400011: Bad Request
                There was a problem publishing the file

[kykrueger]

The problem with the threaded request seems to be a race condition. While debugging, I'd added a breakpoint here: https://github.com/tableau/server-client-python/blob/307d8a20a30f32c1ce615cca7c6a78b9b9bff081/tableauserverclient/server/endpoint/endpoint.py#L109-L112

, and the upload ran fine. Increasing the sleep time also hid the problem.

[kykrueger]

I've created a PR to fix the issues we were having. The use of a custom datetime module caught me off guard in the project. I'd suggest using datetime and timedelta as the source for seconds and minutes instead of manually keeping track of what the seconds and minutes ought to be, but didn't want to make any more changes than necessary to fix the problem.

[kykrueger]

To push this along I've added a check to see if exceptions are returned from the blocking request. This was already returned before this PR, but the type hinting didn't reveal the possibility.

When making my changes, I fixed a pyright error where it had complained that Exceptions could be returned, but were not hinted. This broke the checks for mypy.

Now the returned exceptions will be raised. Before a typeerror would have probably occurred, but now the proper exception will be raised.

[robbertvs]

With tableauserverclient v0.28 and tableau server 2023.1.7 we encountered a similar problem, except TSC did not throw an error. The logs showed that the uploaded chunks were just about 100 bytes:

Publishing <filename>.hyper to server with chunking method (datasource over 64MB, chunk size 50MB)
Initiated file upload session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:30:39 Published 0.00010013580322265625MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:30:42 Published 0.0002002716064453125MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:30:44 Published 0.00030040740966796875MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
...
06:33:38 Published 0.0062999725341796875MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:33:40 Published 0.006400108337402344MB
Uploading a chunk to session (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
06:33:42 Published 0.006428718566894531MB
File upload finished (ID: 10764:bd9bb960e58c4f8eac7360600a89705b-1:0)
Published <filename>.hyper (JOB_ID: 050ce884-adf5-4c69-b93d-18b1a3afc2c7)

We resolved this issue by downgrading to 0.25

[vizyourdata]

I am seeing the same error in tableauserverclient==0.28

Versions Tableau Online Python 3.9.13 tableauserverclient==0.28

My hyper file was very small. The file published successfully and then threw the same error 400011: Bad Request There was a problem publishing the file 'my.hyper'

I downgraded to tableauserverclient==0.25 as suggested and no error.

--Update-- I found this at the bottom of the hyperd log file from the version 0.28 run. {"ts":"2023-11-02T09:45:37.803769","k":"connection-communication-failure","v":{"std-exception":{"type":"system_error","message":"No process is on the other end of the pipe.","error-code":233,"error-category":"system","error-message":"No process is on the other end of the pipe."},"source":"CallbackConnectionCheck"}}

[jacalata]

Thanks for doing the work here! You're right about the custom datetime, and I've opened a separate issue to make sure we get that fixed. I'll leave this open for now to make sure the new release actually works for you - please let me know either way.

[kykrueger]

@jacalata , thanks for the update and considering the datetime use I've set up a ticket on our end to update our scripts to the new version. I'll update here if there is a problem! :rocket:

[kykrueger]

@jacalata , we have been running the fix for a few weeks now and have no longer seen this error. Looks like the issue is resolved. Thanks for your support on the merge!