delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.62k stars 1.71k forks source link

[SPARK] Add metric for DV-related errors #3858

Closed zhipengmao-db closed 2 weeks ago

zhipengmao-db commented 2 weeks ago

Which Delta project/connector is this regarding?

Description

It wraps two RoaringBitmapArray APIs readFrom and serializeAsByteArray with two utils function DeletionVectorUtils.serialize and DeletionVectorUtils.deserialize, which will log a delta event when it fails during (de-)serialization to enable monitoring and debugging the error.

How was this patch tested?

Log-only change.

Does this PR introduce any user-facing changes?

No.