search

ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.11k stars 5.6k forks source link

[rllib] Make results-dir/ logdir prefix consistent #2855

Closed praveen-palanisamy closed 5 years ago

praveen-palanisamy commented 6 years ago

Describe the problem

Training using the Rllib Python API generates results and logs with the date (Y-m-d_H-M-S) as the default dir/file name prefix which makes it difficult to identify/track different runs (requires looking into the json/csv files)

The command line API and the tune API make use of the experiment name which is nice but they do so in different ways (Tune API uses the Experiment spec & the rllib train interface takes it as a command line arg).

Can we make it all consistent and use a single config variable to specify the prefix? If not, at least the default logdir_prefix used by the Agent interface could be changed to include the agent name and/or the env name so that it is much more identifiable/useful than something like this: 2018-09-10_19-10-19nsga6q5w

Not high priority. Good first issue.

ericl commented 6 years ago

@praveen-palanisamy , do you have some examples of what more consistent defaults would look like?

praveen-palanisamy commented 6 years ago

The default logdir prefix could look like this: PPO_Pendulum-v0_2018-09-10_19-10 instead of the current default: 2018-09-10_19-10

praveen-palanisamy commented 6 years ago

Using one config variable (experiment_name) to specify the experiment name that is consistent with all the APIs (command line, tune, python) may be a good idea but will require some refactoring/deprecation of the existing experiment name specifiers. This part could probably be combined with #1738

richardliaw commented 5 years ago

This is also addressed more comprehensively in #3465.