Closed alexjbush closed 5 years ago
Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
---|---|---|---|---|---|
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/FlowContext.scala | 2 | 5 | 40.0% | ||
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkActions.scala | 6 | 9 | 66.67% | ||
<!-- | Total: | 34 | 40 | 85.0% | --> |
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkDataFlow.scala | 2 | 82.26% | ||
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkActions.scala | 2 | 70.37% | ||
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/DataFlow.scala | 3 | 94.03% | ||
<!-- | Total: | 7 | --> |
Totals | |
---|---|
Change from base Build 400: | 0.5% |
Covered Lines: | 1138 |
Relevant Lines: | 1440 |
Description
This change introduces the option of staging a commit with a numeric repartition value.
Also introduces a get conf mechanism on the flow context, and moves the cache commit option to a global conf.
Fixes #53
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Unit tests
Note
The wiki pages for commit actions and parameters will need to be updated