Closed vavison closed 5 years ago
Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
---|---|---|---|---|---|
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/AuditTableFile.scala | 19 | 21 | 90.48% | ||
<!-- | Total: | 22 | 24 | 91.67% | --> |
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
waimak-hive/src/main/scala/com/coxautodata/waimak/metastore/HiveDBConnector.scala | 1 | 96.0% | ||
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/AuditTableFile.scala | 1 | 94.92% | ||
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/StorageActions.scala | 1 | 93.27% | ||
waimak-experimental/src/main/scala/com/coxautodata/waimak/spark/app/SparkApp.scala | 1 | 90.91% | ||
<!-- | Total: | 4 | --> |
Totals | |
---|---|
Change from base Build 537: | -0.01% |
Covered Lines: | 1255 |
Relevant Lines: | 1567 |
Description
Storage compaction no longer performs
hot -> cold
followed bycold -> cold
compactions, instead compacting all of the hot regions plus the cold regions under the configured row threshold in a single go.This reduces complexity, and removes the additional round of IOPs and Spark stage.
Fixes #71
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Existing unit tests are still passing