search

salesforce / TransmogrifAI

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning
https://transmogrif.ai
BSD 3-Clause "New" or "Revised" License
2.24k stars 393 forks source link

Make model selector metadata to metric more robust #386

Closed erica-chiu closed 5 years ago

erica-chiu commented 5 years ago

Related issues

Describe the proposed solution Change the matching when parsing model selector metadata into metrics from name of metric to metric keys

Describe alternatives you've considered N/A

Additional context Allows for parsing of metadata to be based on content rather than the name, allowing for multiple metrics of the same type

codecov[bot] commented 5 years ago

Codecov Report

Merging #386 into master will increase coverage by <.01%. The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #386      +/-   ##
==========================================
+ Coverage   86.85%   86.86%   +<.01%     
==========================================
  Files         336      336              
  Lines       10950    10956       +6     
  Branches      351      578     +227     
==========================================
+ Hits         9511     9517       +6     
  Misses       1439     1439
Impacted Files Coverage Δ
...op/stages/impl/selector/ModelSelectorSummary.scala 93% <100%> (+0.44%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 07a4a73...79c9083. Read the comment docs.

tovbinm commented 5 years ago

@erica-chiu proposed tests are defining the custom evaluator incorrectly. Here is how is supposed to work #387