search

geyang / ml_logger

A logger, server and visualization dashboard for ML projects
Other
192 stars 17 forks source link

Filtering by hyperparameters in ml-dash #27

Open btickell opened 4 years ago

btickell commented 4 years ago

I'm trying to revamp my workflow and this package seems to be a good starting point. I was wondering if it is possible to filter results based on parameters configs. I am running experiments with doodad and as such I have several seeds under a single experiment-name. I want to grab values based on either unique parameter configurations or results that share the same value (lr=.01 for example). Is this currently available?

geyang commented 4 years ago

Yes, you can take a look at the examples here:

https://ml-logger.readthedocs.io/en/latest/modindex.html?highlight=get_parameters#ml_logger.ML_Logger.get_parameters

experiments = logger.glob("some_prefix/blah/*/parameters.pkl")
for exp_key in experiments:
      params = logger.get_params(key=exp_key)

I just typed this up by hand, you can check the docs for details.

btickell commented 4 years ago

Ok this looks good, can I get plots from multiple runs based on this filter or is dynamic plotting something that needs to be done on top?

geyang commented 3 years ago

@btickell Sorry for this late reply! A lot happened in the past year. In case this is helpful, this does return results from all of the experiments that match that glob string. I believe everything is combined in a single DataFrame object.