CoxAutomotiveDataSolutions / waimak

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.
Apache License 2.0
75 stars 16 forks source link

Rework storage API #60

Closed alexjbush closed 5 years ago

alexjbush commented 5 years ago

Description

This PR makes several breaking changes to improve storage actions. Audit tables are now entities on the flow, and all audit table reading happens in actions on the flow.

Fixes #51

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

Unit tests

alexjbush commented 5 years ago

Working on scaladocs

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 421


Changes Missing Coverage Covered Lines Changed/Added Lines %
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/StorageActions.scala 30 31 96.77%
<!-- Total: 52 53 98.11% -->
Files with Coverage Reduction New Missed Lines %
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/FileStorageOps.scala 1 92.31%
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/AuditTableFile.scala 1 95.98%
<!-- Total: 2 -->
Totals Coverage Status
Change from base Build 411: 0.4%
Covered Lines: 1148
Relevant Lines: 1445

💛 - Coveralls