search

byu-dml / d3m-experimenter

A distributed system for creating, running, and persisting many machine learning experiments.
0 stars 0 forks source link

`all_metrics` parameter doesn't score against metrics listed in problem description when set to False #93

Closed e13h closed 4 years ago

e13h commented 4 years ago

In execute_pipeline_on_problem(), the docstring for the parameter all_metrics says the following:

https://github.com/byu-dml/d3m-experimenter/blob/001a68aa988d58cecf2de078ce6bdc72c266641b/experimenter/execute_pipeline.py#L39-L41

But when all_metrics is False, this is not actually the case. The metrics used when all_metrics is False is None. https://github.com/byu-dml/d3m-experimenter/blob/001a68aa988d58cecf2de078ce6bdc72c266641b/experimenter/execute_pipeline.py#L57-L59

e13h commented 4 years ago

Nevermind, I was wrong. https://github.com/byu-dml/d3m-experimenter/blob/001a68aa988d58cecf2de078ce6bdc72c266641b/experimenter/run_pipeline.py#L132-L135