search

databricks / databricks-sql-go

Golang database/sql driver for Databricks SQL.
Apache License 2.0
34 stars 37 forks source link

staging operation over HTTP was unsuccessful: <HeaderName>Transfer-Encoding</HeaderName> #222

Closed elise-prequel closed 2 months ago

elise-prequel commented 2 months ago

Hi! This might be an issue with https://github.com/databricks/databricks-sdk-go; opening here because we saw some changes to staging operations in the latest version release notes on this package, and nothing particularly suspicious in the other.

Description We upgraded our versions from

github.com/databricks/databricks-sdk-go v0.37.0 // indirect
github.com/databricks/databricks-sql-go v1.5.3 // indirect

to the latest version of both packages.

Our exact query is:

PUT '{{.LocalFilePath}}' INTO '{{.UnityVolumePath}}' OVERWRITE;

and we get an error about the Transfer-Encoding header not being accepted, full error is

databricks: driver error: staging operation over HTTP was unsuccessful: 400-<?xml version="1.0" encoding="utf-8"?><Error><Code>UnsupportedHeader</Code><Message>One of the HTTP headers specified in the request is not supported.
RequestId:cac6b3df-d01e-0000-1e66-9abf07000000
Time:2024-04-29T18:50:37.3231103Z</Message><HeaderName>Transfer-Encoding</HeaderName></Error>

Reproduction We think it is any PUT with the golang setup. We tested on our regular instance (SQL endpoint on the latest release channel) and on Azure.

Expected behavior Normal PUT.

Is it a regression? Yes, downgrading to 1.5.3 fixed our issue.

kravets-levko commented 2 months ago

Hi @elise-prequel! The fix for this issue will be available in upcoming v1.5.5 (databricks/databricks-sql-go#219) very soon

elise-prequel commented 2 months ago

Thank you so much @kravets-levko !!

niger-prequel commented 2 months ago

@kravets-levko would you mind explaining which change in v1.5.5. addresses this? based on the change log it would appear to be #217 but that seems to be focusing on the Content-Size header rather than Transfer-Encoding. Is it just the case that not having the correct Content-Size header ends up resulting in an error about Transfer-Encoding?

kravets-levko commented 2 months ago

@niger-prequel Yes, I think that it's related to databricks/databricks-sql-go#217 - there are no much changes in 1.5.4 comparing to 1.5.3, so it should be the one

niger-prequel commented 2 months ago

Ok @kravets-levko , I just want to ensure that this is not related to github.com/databricks/databricks-sdk-go jumping from v0.37.0 to v0.39.0. Because if it is and we upgrade, we'll encounter the same issue.

kravets-levko commented 2 months ago

@niger-prequel @elise-prequel v1.5.5 is out - please give it a try and let me know if you still see this error

niger-prequel commented 2 months ago

@kravets-levko it upgrading to v1.5.5 passed our test suite

kravets-levko commented 2 months ago

Great, I'm glad to hear that. I'm closing the issue, but feel free to re-open if needed, or open a new one