alexjbush
commented
6 years ago Would be nice to put a screenshot and a few lines in the readme since it's such a neat feature
Closed dJalon closed 5 years ago
alexjbush
commented
6 years ago Would be nice to put a screenshot and a few lines in the readme since it's such a neat feature
coveralls
commented
6 years ago | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
|---|---|---|---|---|---|
| waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/FlowReporter.scala | 1 | 2 | 50.0% | ||
| waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkFlowReporter.scala | 2 | 3 | 66.67% | ||
| waimak-core/src/main/scala/org/apache/spark/ui/WaimakExecutionEvent.scala | 3 | 4 | 75.0% | ||
| waimak-core/src/main/scala/org/apache/spark/ui/WaimakExecutionsUITab.scala | 8 | 9 | 88.89% | ||
| waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/ui/WaimakGraph.scala | 36 | 39 | 92.31% | ||
| waimak-core/src/main/scala/org/apache/spark/ui/WaimakExecutionsUIPage.scala | 0 | 13 | 0.0% | ||
| waimak-core/src/main/scala/org/apache/spark/ui/WaimakExecutionUIPage.scala | 0 | 18 | 0.0% | ||
| waimak-core/src/main/scala/org/apache/spark/ui/WaimakGraphRenderer.scala | 0 | 20 | 0.0% | ||
| <!-- | Total: | 62 | 120 | 51.67% | --> |
| Files with Coverage Reduction | New Missed Lines | % | ||
|---|---|---|---|---|
| waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/DataFlow.scala | 1 | 97.65% | ||
| waimak-impala/src/main/scala/com/coxautodata/waimak/metastore/ImpalaDBConnector.scala | 1 | 69.57% | ||
| waimak-storage/src/main/scala/com/coxautodata/waimak/storage/FileStorageOps.scala | 1 | 91.38% | ||
| waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkDataFlow.scala | 1 | 86.67% | ||
| waimak-storage/src/main/scala/com/coxautodata/waimak/storage/StorageActions.scala | 3 | 85.11% | ||
| <!-- | Total: | 7 | --> |
| Totals | |
|---|---|
| Change from base Build 246: | -2.6% |
| Covered Lines: | 964 |
| Relevant Lines: | 1300 |
vavison
commented
6 years ago There seems to be an issue with none of the flow-related jobs appearing in the job or stage tabs of the history server (everything is fine in the Spark UI while the job is running).
Weird one - needs investigating
alexjbush
commented
6 years ago @alexeipab What do you think?
alexjbush
commented
5 years ago Closing for now, needs more thinking about how best to lay out the DAG.
Description
New feature added. In the Spark ui, Waimak now adds a tab displaying a DAG.
Type of change
How Has This Been Tested?
This has been tested visually to spot inaccurate DAGs displayed in the web interface. Several unit tests have been left written to cover the correct functionality of the new functions added.