Adopt or not to adopt tensorboard native hyperparameters recording

[vwxyzjn]

Problem Description

The current way of adding hyperparameters is to do it via markdown text:

writer = SummaryWriter(f"runs/{run_name}")
writer.add_text(
    "hyperparameters",
    "|param|value|\n|-|-|\n%s" % ("\n".join([f"|{key}|{value}|" for key, value in vars(args).items()])),
)

It is however to use the tensorboard native hyperparameters plugin, which unfortunately also has ugly code... see https://github.com/pytorch/pytorch/issues/37738#issuecomment-1124497827

writer = SummaryWriter(f"runs/{run_name}")
from torch.utils.tensorboard.summary import hparams
exp, ssi, sei = hparams(vars(args), metric_dict={"charts/episodic_return": 0})   
writer.file_writer.add_summary(exp)                 
writer.file_writer.add_summary(ssi)                 
writer.file_writer.add_summary(sei)  

produces the following demo, which I guess is pretty cool

https://user-images.githubusercontent.com/5555347/167988018-50c94e3f-14bc-4af9-bebb-24e95d9cd2fb.mov

Checklist

• [x] I have installed dependencies via poetry install (see CleanRL's installation guideline.
• [x] I have checked that there is no similar issue in the repo (required)

CC @araffin and @Miffyli and who might be interested in this hidden feature.

[araffin]

You may have a look at https://github.com/DLR-RM/stable-baselines3/issues/428#issuecomment-1124815614 too