ryanjulian / rllab

rllab is a framework for developing and evaluating reinforcement learning algorithms, fully compatible with OpenAI Gym.
Other
1 stars 21 forks source link

Add save graph and tensor logging to tensorboard #88

Closed CatherineSue closed 6 years ago

CatherineSue commented 6 years ago

Add a customized tensor scalar to tensorboard by using the custom_scalar plugin in tensorboard. Each line in the scalar corresponds to an element in the tensor.

Wrap the tensorboard logging module into a new class Summary in file rllab/misc/tensor_summary.py. It supports both the simple value and tensor logging. It also saves the computation graph created by rllab.

To record the tensor into tensorboard, use the record_tensor function in file rllab/misc/logger.py.

Refer to: #39, #38

CatherineSue commented 6 years ago

The test result of the custom tensor scalar is shown in the following image. The scalar groups tensors by their name_scope. tensorboard

CatherineSue commented 6 years ago

Changed member names of class Summary in file rllab/misc/tensor_summary.py in order to be consistent with the class in #58.

CatherineSue commented 6 years ago

Rebased the repo and refactored the code in tensorboard_output.py. Sorry about the earlier messy commits.

Changes I made:

  1. Regroup the imports
  2. Change the class name Summary to TensorBoardOutput.
  3. Refactor the name of the class member. Set the private member function names to _*, majorly the dump function.
  4. Remove unrequired class member.
  5. Preserve the tensor dimensions and iterate over dimensions. Make the tag name to i,j,k,... Can't include brackets in the tag of scalars because of tensorboard cursom_scalar doesn't support them
CatherineSue commented 6 years ago

Needs improvement:

  1. There will be scalars in both tab Scalar and Customized Scalar in tensorboard. Tried to find a way to show tensors only in Customized Scalar but failed.
  2. The custom_scalar plugin don't group the same layout together. In a result, if we have a log_dir which has several runs, the result of the customized scalar is like this: screen shot 2018-05-25 at 4 54 59 pm

    There are two trig/bar:0 scalars.

ryanjulian commented 6 years ago

Hmm okay. Investigate and let me know what you find out. Done is better than perfect :)

CatherineSue commented 6 years ago

The custom_scalar plugin firstly writes the layout proto into a custom_scalar_config inside the run file. As a result, each run has a custom_scalar_config in its run file. When we start tensorboard at the top-level dir, it recursively finds runs within the tensorboard_dir. Thus, there will be duplicate charts, like the one above.

It seems that the original designer doesn't support the idea of each config per run (https://github.com/tensorflow/tensorboard/pull/664):

(I think per-run config files should be ignored ... what would a per-run config file mean?)

To deal with this issue, I pertain a global custom_scalar_config in the top-level dir. When there is a new run started, I merge the new run's layout proto into the global layout proto, so that there will be no duplicate fields anymore. When tensorboard starts, it only gets one custom_scalar_config.

e.g. The file inside the custom_scalar_config contains the global custom_scalar_config.

screen shot 2018-05-26 at 1 15 06 am

And the tensorboard result is like this.

screen shot 2018-05-26 at 1 15 31 am
ryanjulian commented 6 years ago

Great job!