[Spark][3.2] Fix CommitInfo.inCommitTimestamp deserialization for very small timestamps - Githubissues

delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

https://delta.io

Apache License 2.0

6.96k stars 1.58k forks source link

[Spark][3.2] Fix CommitInfo.inCommitTimestamp deserialization for very small timestamps #3045

Closed dhruvarya-db closed 1 week ago

dhruvarya-db commented 2 weeks ago

Which Delta project/connector is this regarding?

[X] Spark
[ ] Standalone
[ ] Flink
[ ] Kernel
[ ] Other (fill in here)

Description

Currently, if we deserialize a CommitInfo with a very small inCommitTimestamp and then try to access this inCommitTimestamp, this exception is thrown

java.lang.ClassCastException: class java.lang.Integer cannot be cast to class java.lang.Long

This PR fixes the CommitInfo so that the inCommitTimestamp field is deserialized correctly.

How was this patch tested?

Added a new test case that was failing before the fix.

Does this PR introduce any user-facing changes?

No