search

salesforce / TransmogrifAI

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
https://transmogrif.ai
BSD 3-Clause "New" or "Revised" License
2.24k stars 392 forks source link

Avoid hardcoded default evaluators in ModelSelectors #497

Closed nicodv closed 4 years ago

nicodv commented 4 years ago

Related issues N/A

Describe the proposed solution Since the model selectors hardcoded the default evaluators, we lacked the ability to set parameters on these evaluators there. This has been fixed by simply setting the evaluators as default arguments. This change was made to be backward compatible by prepending the defaults to the Seq of evaluators if they're not there.

Describe alternatives you've considered You can add a duplicate of the default evaluator with different parameters to the list of evaluators, but you'd have to separate them again later on (and they'd be same class and name etc.).

Additional context Related to https://github.com/salesforce/TransmogrifAI/pull/496, which introduces several parameters on OpRegressionEvaluator

codecov[bot] commented 4 years ago

Codecov Report

Merging #497 into master will increase coverage by 14.28%. The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #497       +/-   ##
===========================================
+ Coverage   72.72%   87.00%   +14.28%     
===========================================
  Files         345      345               
  Lines       11702    11706        +4     
  Branches      388      371       -17     
===========================================
+ Hits         8510    10185     +1675     
+ Misses       3192     1521     -1671
Impacted Files Coverage Δ
...sification/BinaryClassificationModelSelector.scala 98.46% <100.00%> (+0.04%) :arrow_up:
...ssification/MultiClassificationModelSelector.scala 97.56% <100.00%> (+82.92%) :arrow_up:
...ages/impl/regression/RegressionModelSelector.scala 98.14% <100.00%> (ø)
...op/stages/impl/selector/ModelSelectorFactory.scala 88.88% <100.00%> (+3.17%) :arrow_up:
...es/src/main/scala/com/salesforce/op/OpParams.scala 85.71% <0.00%> (-4.09%) :arrow_down:
...com/salesforce/op/features/types/FeatureType.scala 95.95% <0.00%> (+1.01%) :arrow_up:
...scala/com/salesforce/op/features/FeatureLike.scala 43.47% <0.00%> (+1.08%) :arrow_up:
...rce/op/stages/impl/preparators/SanityChecker.scala 90.57% <0.00%> (+1.63%) :arrow_up:
...a/com/salesforce/op/filters/RawFeatureFilter.scala 92.97% <0.00%> (+2.16%) :arrow_up:
...scala/com/salesforce/op/testkit/RandomStream.scala 100.00% <0.00%> (+2.38%) :arrow_up:
... and 124 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 7759915...9900e6a. Read the comment docs.