delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
https://delta.io
Apache License 2.0
7.22k stars 1.62k forks source link

[DELTA] Add logs around managed commits #3246

Closed prakharjain09 closed 2 weeks ago

prakharjain09 commented 2 weeks ago

Which Delta project/connector is this regarding?

Description

Add logging around managed commits.

How was this patch tested?

UTs

Does this PR introduce any user-facing changes?

No

scottsand-db commented 2 weeks ago

Seems to be a flaky (timeout) test. We can force merge this.

https://github.com/delta-io/delta/actions/runs/9457025237/job/26050028706?pr=3246

[info] - PredicatePushdown: Multiple delete statements. vectorizedReaderEnabled: false readColumnarBatchAsRows: false
[info] - PredicatePushdown: Scan with predicates. vectorizedReaderEnabled: true readColumnarBatchAsRows: true *** FAILED ***
[info]   The code passed to failAfter did not complete within 20 minutes. (DeletionVectorsSuite.scala:1022)
[info]   org.scalatest.exceptions.TestFailedDueToTimeoutException:
[info]   at java.lang.Thread.getStackTrace(Thread.java:1564)
[info]   at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:277)
[info]   at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
[info]   at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
[info]   at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:69)
[info]   at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:155)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)