How do I plot the success rate in tensorboard? [question]

[qiqqx]

I successfully implemented tensorboard in the code and now i want to log the success rate.

[araffin]

Hello, Please fill the issue template completely, a lot of context is missing... But you have probably the answer in the documentation

[qiqqx]

Thank you for replying. I used the TensorboardCallback but I dont know how to get the value of the FetchReach-v1 environment without resetting it.

value = np.random.random()
        summary = tf.Summary(value=[tf.Summary.Value(tag='random_value', simple_value=value)])
        self.locals['writer'].add_summary(summary, self.num_timesteps)
        return True

How do I get the "done" value without resetting the environment?

[araffin]

I used the TensorboardCallback but I dont know how to get the value of the FetchReach-v1 environment without resetting it.

Again, some context is missing. I assume you are using HER. In that case, we already log the success rate using the logger. (it is available in the callback using self.logger) You can display that value using the "legacy" integration (cf doc). A better method would be to use an EvalCallback and compute the success rate there.

[qiqqx]

Thank you very much 👍

[DinisMoreira]

Hello, I think I have the same problem. I'm also using HER, but I did not quite understand @araffin 's answer. How do I get the "done" value from the EvalCallback?

[Miffyli]

@DinisMoreira I am not sure why you would want to get "done" value from EvalCallback. EvalCallback itself does the evaluation after every Nth number of steps, and you do not need to worry about other things if it is evaluation performance you want.

[DinisMoreira]

Sorry, I'm still new to ML and navigating my way through. My end goal was to plot additional values in tensorboard, like some kind of "average success rate" from the last 100 episodes. I was trying yo make a Custom callback but I'm having some trouble regarding how to find out from the custom callback if the episode was successful or not, or find out a better way. I'm using HER (TD3)

[Miffyli]

I recommend migrating to stable-baselines3 which has better support for setups like this (and more documentation on logging things to tensorboard). I have not used tensorboard myself, but it should already have "average reward" curve which tells you about the success rate.

[araffin]

I recommend migrating to stable-baselines3 which has better support for setups like this (and more documentation on logging things to tensorboard). I have not used tensorboard myself, but it should already have "average reward" curve which tells you about the success rate.

like some kind of "average success rate" from the last 100 episodes. I

This is done in SB3, both for training and EvalCallback: https://github.com/DLR-RM/stable-baselines3/blob/a038044d11d7d5e4676c512c1561e9d3d33d6713/stable_baselines3/common/callbacks.py#L415

[DinisMoreira]

Thank you so much!