add wandb.log feature to record episode log

[PatrickYanZ]

Description:

The current wandb log is recorded by the writer. It is not easy to extract data from the log text file. To extend the usage for plot and

Ref: https://docs.wandb.ai/guides/track/log

Change:

1. add wandb.define_metric("train/step")
2. create a log_dict in __report
3. log each entry in wandb. wandb.log(log_dict)

[ffelten]

Hi, thanks for contributing 🔥.

However, I'm not sure I understand the purpose of this. We usually log against global_step e.g. https://github.com/LucasAlegre/morl-baselines/blob/main/morl_baselines/common/utils.py#L171. How is this PR different from that?

Also, in my experience, getting the data out of wandb through their API is doable. Can you explain a bit more what is the issue here?

[PatrickYanZ]

Hi @ffelten , thanks for your reply. Once training is finished, I expect to see the data to be downloaded or through API call to retrieve. But Once I log in to the wandb as also instructed. I cannot find the data in the board or in somewhere else (i.e. host pc). Only if I record the log, I can do the visualization plot and analysis for the next step.

ref: https://wandb.ai/site/articles/export-data-from-wb https://docs.wandb.ai/guides/track/log

The only log I have is the plain text log is output.log.

Episode infos:
Steps: 30, Time: 2.2099609375
Total Reward: [ 8.575505 19.5       0.      ], Discounted: [ 7.1515403 15.872447   0.       ]
Scalarized Reward: 12.461659262238548, Discounted: 10.253435835563735

But the problem is, it is hard to extract the data from it. and the file is so long that some scripts are required to read and analysis the log.

I do not know if is there anyway for you to retrieve the log information, so I raise this pull request. However, please let me aware if you have any better ideas.

[ffelten]

Hi @ffelten , thanks for your reply. Once training is finished, I expect to see the data to be downloaded or through API call to retrieve. But Once I log in to the wandb as also instructed. I cannot find the data in the board or in somewhere else (i.e. host pc). Only if I record the log, I can do the visualization plot and analysis for the next step.

ref: https://wandb.ai/site/articles/export-data-from-wb https://docs.wandb.ai/guides/track/log

Do you log locally i.e. using wandb server start or do you log remotely i.e. to their server?

The only log I have is the plain text log is output.log.

Episode infos:
Steps: 30, Time: 2.2099609375
Total Reward: [ 8.575505 19.5       0.      ], Discounted: [ 7.1515403 15.872447   0.       ]
Scalarized Reward: 12.461659262238548, Discounted: 10.253435835563735

But the problem is, it is hard to extract the data from it. and the file is so long that some scripts are required to read and analysis the log.

I do not know if is there anyway for you to retrieve the log information, so I raise this pull request. However, please let me aware if you have any better ideas.

What I do in general is that I run my code on a server which sends the log to my wandb account (on their server). So I can see the data in real time via the wandb UI. And I can extract the log using their python API afterwards to make plots for papers. e.g. https://docs.wandb.ai/guides/track/public-api-guide#read-specific-metrics-from-a-run

For example, I extract hypervolume with code similar to that:

api = wandb.Api()

runs = api.runs("<entity/project-name>") # replace with your thing e.g. "florianfelten/MORL-Baselines"
for run in runs:
    df = run.history(keys=["charts/hypervolume"], x_axis="global_step").dropna()
    # do whatever you wish with the panda dataframe...

Note: you must configure your env variables with your WANDB_API_KEY and WANDB_BASE_URL if you use their server.

[PatrickYanZ]

Thanks a lot @ffelten. Yes, I log in remotely to their server. Also, I check the log on the local.

First, I cannot find any graphs and data result on the server. I can only see my system information on their server (i.e. CPU consumption). I cannot find any otherl og data or training data remotely. Like I state before, I only see the output.log plain text in the log.

import wandb
api = wandb.Api()

run = api.run("york-ngwn/MORL-Baselines/x7bduq5s")
print(run.config)
print(run.summary)
df = run.history().dropna()
print(df)

The output is

{'per': True, 'tau': 1, 'gamma': 0.98, 'env_id': 'mo-highway-fast-v0', 'net_arch': [256, 256], 'batch_size': 64, 'buffer_size': 2000000, 'num_sample_w': 4, 'use_envelope': True, 'learning_rate': 0.0003, 'clip_grand_norm': 0.1, 'initial_epsilon': 1, 'learning_starts': 100, 'gradient_updates': 1, 'epsilon_decay_steps:': 50000, 'homotopy_decay_steps': 10000, 'final_homotopy_lambda': 1, 'target_net_update_freq': 1000, 'initial_homotopy_lambda': 0}
{'_wandb': {'runtime': 9069}}
Empty DataFrame
Columns: []
Index: []

[ffelten]

Closing as it does not fix the issue. We will have a look why the data are not visible in wandb for you.