delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
6.96k stars 1.58k forks source link

[Spark] Extend Fuzz test For Managed Commit #3049

Closed junlee-db closed 1 week ago

junlee-db commented 2 weeks ago

Which Delta project/connector is this regarding?

Description

This PR extends Fuzz test to test managed commit features. Specifically, it adds a new event phase inside commit operation, so that we can capture the backfill as a separate operation. By doing so, it is possible that multiple commits can go through before backfill and managed commit is expected to deal with various situation to return the correct output.

How was this patch tested?

Existing fuzz tests should naturally use the extended backfill phases.

Does this PR introduce any user-facing changes?

No