search

salesforce / TransmogrifAI

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
https://transmogrif.ai
BSD 3-Clause "New" or "Revised" License
2.24k stars 393 forks source link

Aggregate spark metrics during run time instead of post processing by default #358

Closed gerashegalov closed 5 years ago

gerashegalov commented 5 years ago

Replaces the previous PR #341

salesforce-cla[bot] commented 5 years ago

Thanks for the contribution! It looks like @rachelwarren is an internal user so signing the CLA is not required. However, we need to confirm this.

tovbinm commented 5 years ago

Scalastyle problem: 

> Task :utils:scalaStyle
warning file=/home/travis/build/salesforce/TransmogrifAI/utils/src/main/scala/com/salesforce/op/utils/spark/OpSparkListener.scala message=Use correct indentation line=323
error file=/home/travis/build/salesforce/TransmogrifAI/utils/src/test/scala/com/salesforce/op/utils/spark/OpSparkListenerTest.scala message=Expected token SEMI but got Token(INTERPOLATION_ID,s,4046,s)
codecov[bot] commented 5 years ago

Codecov Report

Merging #358 into master will increase coverage by 0.09%. The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #358      +/-   ##
==========================================
+ Coverage   86.72%   86.82%   +0.09%     
==========================================
  Files         336      336              
  Lines       10799    10865      +66     
  Branches      346      568     +222     
==========================================
+ Hits         9365     9433      +68     
+ Misses       1434     1432       -2
Impacted Files Coverage Δ
...om/salesforce/op/utils/spark/OpSparkListener.scala 98.6% <100%> (+1.19%) :arrow_up:
...es/src/main/scala/com/salesforce/op/OpParams.scala 89.79% <0%> (+4.08%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3b82d96...900cec5. Read the comment docs.