use tensorboard to monitor training/testing

[AlexQuant62]

subj. instead of TT renderers. if using Ray, implement custom callback class. see issue/example here: https://github.com/ray-project/ray/issues/7871

[rhamnett]

Please provide a specific TT example (use the ray tutorial as a base if you require) https://www.tensortrade.org/en/latest/tutorials/ray.html

[AlexQuant62]

I created a notebook: https://github.com/AlexQuant62/test01/blob/main/ray_with_callbacks.ipynb

Class definition for callbacks from: https://github.com/ray-project/ray/blob/0c80efa2a37f482494fbffbe9e81f61586b03ecb/rllib/examples/custom_metrics_and_callbacks.py

• Modified to add custom metrics for TensorBoard (max/mean/min) : net_worth, num_batches (added by default).
• Debug prints commented.
• During debugging it was found that sometimes episode.last_info_for() return None
• Don`t know whether it worth it to add these metrics, in my testing (DQN) i receive warnings that 'process_trial' and 'experiment_checkpoint' operations take 1-2 sec to complete on each iteration which more than double training time.

[rhamnett]

@AlexQuant62 Hi this looks like some code I wrote a while back. Did you find this somewhere? Just curious as I have got a more comprehensive version I can share.

[AlexQuant62]

@rhamnett Hi, no didn't find your code. Please, share. I'm more than less novice in Python ;)

[rhamnett]

Will do, just waiting for a bug that I reported to be fixed in Ray :)

[carlogrisetti]

@rhamnett can you link the issue in Ray?

[carlogrisetti]

@AlexQuant62 did you have a look at the most recent example? https://www.tensortrade.org/en/latest/examples/train_and_evaluate_using_ray.html

However the thing you are suggesting is one of the things I would want to implement in the future, having a "balance" or "net_worth" value logged in tensorboard, that shows the "human" outcome of the training. This can be useful in training, but can be even more useful in evaluation. As an added bonus, to leverage tensortrade hyperparameter optimization tab (HPARAMS should be the title if I'm not mistaken), I would like to log the maximum reward value it reached (going on the idea that you have that checkpoint saved, and you can use that)

[carlogrisetti]

Couldn't manage to get some sort of custom metric in the evaluation stage. It can be done but during the training itself, not during evaluation, since there's no "on_evaluation_end" or similar callback in Ray.

Will try to look into that a little bit more