apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
https://kyuubi.apache.org/
Apache License 2.0
2.08k stars 907 forks source link

[Bug] kyuubi not support UTC+8 spark.sql.session.timeZone format #5219

Closed user-GitHub-user closed 1 year ago

user-GitHub-user commented 1 year ago

Code of Conduct

Search before asking

Describe the bug

kyuubi version: 1.7.1 spark version: 3.2.1 I configure spark.sql.session.timeZone=UTC+8 in spark-default.conf under the spark conf directory to specify the time zone In this case, the data queried by kyuubi is 8 hours less than the data queried by Spark-SQL. If I change the configuration to spark.sql.session.timeZone=+08:00 , the data queried on both sides is the same. I guess kyuubi doesn't support the UTC+8 time zone description format

Affects Version(s)

1.7.0,1.7.1

Kyuubi Server Log Output

No response

Kyuubi Engine Log Output

No response

Kyuubi Server Configurations

No response

Kyuubi Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

github-actions[bot] commented 1 year ago

Hello @user-GitHub-user, Thanks for finding the time to report the issue! We really appreciate the community's efforts to improve Apache Kyuubi.

pan3793 commented 1 year ago

Description of spark.sql.session.timeZone from Spark docs https://spark.apache.org/docs/latest/configuration.html

The ID of session local timezone in the format of either region-based zone IDs or zone offsets. Region IDs must have the form 'area/city', such as 'America/Los_Angeles'. Zone offsets must be in the format '(+|-)HH', '(+|-)HH:mm' or '(+|-)HH:mm:ss', e.g '-08', '+01:00' or '-13:33:33'. Also 'UTC' and 'Z' are supported as aliases of '+00:00'. Other short names are not recommended to use because they can be ambiguous.

Seems by design, does Aisa/Shanghai work expected?

user-GitHub-user commented 1 year ago

Aisa/Shanghai is no problem

Using UTC+8 is to avoid Daylight Saving Time: DST issues

It seems that this problem can be solved by changing it to+08:00

pan3793 commented 1 year ago

@user-GitHub-user According to the docs referenced above, I don't think it's a "problem", it's by design.

user-GitHub-user commented 1 year ago

You're right, but before I used kyuubi, my configuration was UTC+8, and Spark could correctly recognize it. Anyway, I will close this issue