CoxAutomotiveDataSolutions / waimak

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.
Apache License 2.0
75 stars 16 forks source link

Feature/better interceptor description #17

Closed dJalon closed 6 years ago

dJalon commented 6 years ago

Description

The intercepted action description has been extended. It now looks something like the following, for an intercepted action with a transform and cache intercept:

Action: PostActionInterceptor Inputs: [example_in] Outputs: [example_out]
Intercepted Action: TestAction Inputs: [example_in] Outputs: [example_out]
Intercepted with: TransformPostAction on [example_out], CachePostAction on [example_out]

Fixes # (issue)

Information about the underlying intercepted is no longer lost.

Type of change

Please delete options that are not relevant.

How Has This Been Tested?

A unit test has been added.

coveralls commented 6 years ago

Pull Request Test Coverage Report for Build 126


Files with Coverage Reduction New Missed Lines %
waimak-rdbm-ingestion/src/main/scala/com/coxautodata/waimak/rdbm/ingestion/RDBMIngestionUtils.scala 1 93.75%
waimak-core/src/main/scala/com/coxautodata/waimak/log/Logging.scala 1 23.08%
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/DataFlow.scala 1 98.63%
waimak-rdbm-ingestion/src/main/scala/com/coxautodata/waimak/rdbm/ingestion/SQLServerTemporalExtractor.scala 1 83.33%
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkInterceptors.scala 1 95.24%
waimak-rdbm-ingestion/src/main/scala/com/coxautodata/waimak/rdbm/ingestion/RDBMIngestionActions.scala 1 94.59%
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/AuditTableFile.scala 2 95.0%
waimak-storage/src/main/scala/com/coxautodata/waimak/storage/StorageActions.scala 2 86.67%
waimak-impala/src/main/scala/com/coxautodata/waimak/metastore/ImpalaDBConnector.scala 2 71.43%
waimak-core/src/main/scala/com/coxautodata/waimak/dataflow/spark/SparkActions.scala 2 69.15%
<!-- Total: 14 -->
Totals Coverage Status
Change from base Build 122: -0.7%
Covered Lines: 769
Relevant Lines: 1031

💛 - Coveralls