[GLUTEN-7028][CH][Part-9] Collecting Delta stats for parquet

baibaichen commented 2 days ago

What changes were proposed in this pull request?

Introducing DeltaStats to collect stats as delta does.

(Fixes: #7028)

How was this patch tested?

Using Existed Uts

In test("test parquet table write with the delta"), adding logic to verify delta stats

    if (spark35) {
      val vanillaTable = "lineitem_delta_parquet_vanilla"
      withSQLConf((GlutenConfig.NATIVE_WRITER_ENABLED.key, "false")) {
        doInsert(drop(vanillaTable), createLineitem(vanillaTable), insert(vanillaTable))
      }
      val expected = DeltaStatsUtils
        .statsDF(
          spark,
          s"$basePath/$vanillaTable/_delta_log/00000000000000000001.json",
          q1SchemaString)
        .collect()

      checkAnswer(
        DeltaStatsUtils.statsDF(
          spark,
          s"$basePath/$table/_delta_log/00000000000000000001.json",
          q1SchemaString),
        expected
      )
    }