search

hill-a / stable-baselines

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
http://stable-baselines.readthedocs.io/
MIT License
4.15k stars 723 forks source link

Store the training result #1169

Closed hotpotking-lol closed 2 years ago

hotpotking-lol commented 2 years ago

Hello, I am new to SB3. I am trying to store the "value_loss" in each update while using "model.learn()".

During training, when verbose = 1, these results will show during each update:

| rollout/ | | | ep_len_mean | 1.33e+03 | | ep_rew_mean | 0.0588 | | time/ | | | fps | 203 | | iterations | 3 | | time_elapsed | 44 | | total_timesteps | 9000 | | train/ | | | approx_kl | 0.01680592 | | clip_fraction | 0.26 | | clip_range | 0.2 | | entropy_loss | -5.67 | | explained_variance | 0.579 | | learning_rate | 0.0003 | | loss | 0.0227 | | n_updates | 20 | | policy_gradient_loss | -0.0167 | | std | 0.994 | | value_loss | 0.00867 |

How can I record these results (maybe in a numpy array?) for the whole training? I would like to have a plot to show how the value loss decrease during the training process. I checked out SB3 website, there is a callback which help me to store the reward. But there is no "value loss" stored at the same path as the reward.

Thank you

Miffyli commented 2 years ago

Hey. Try modifying the logger with these docs. That should give you a csv file for logging stuff.

hotpotking-lol commented 2 years ago

OH! Thank you so much!