kad99kev
commented
2 years ago Hi,
Firstly, thank you for your work on this library.
I have a potential solution to have wandb working with sinergym and sb3. There isn't much to be changed from the example given for MLFlow here.
I have added a minimal working example.
import sys
import gym
import wandb
import numpy as np
from typing import Any, Dict, Tuple, Union
import sinergym
from sinergym.utils.wrappers import LoggerWrapper, NormalizeObservation
from sinergym.utils.constants import RANGES_5ZONE
from sinergym.utils.callbacks import LoggerCallback
from stable_baselines3 import PPO
from stable_baselines3.common.logger import HumanOutputFormat, Logger, KVWriter
# The only addition required.
class WandBOutputFormat(KVWriter):
"""
Dumps key/value pairs onto WandB.
"""
def write(
self,
key_values: Dict[str, Any],
key_excluded: Dict[str, Union[str, Tuple[str, ...]]],
step: int = 0,
) -> None:
for (key, value), (_, excluded) in zip(
sorted(key_values.items()), sorted(key_excluded.items())
):
if excluded is not None and "wandb" in excluded:
continue
if isinstance(value, np.ScalarType):
if not isinstance(value, str):
wandb.log({key: value}, step=step)
# Init wandb here.
wandb.init(
project="project_name",
entity="entity_name",
name="random_agent_" + wandb.util.generate_id(),
)
env = gym.make('Eplus-5Zone-hot-continuous-v1')
env = NormalizeObservation(env, ranges=RANGES_5ZONE)
env = LoggerWrapper(env)
loggers = Logger(
folder=None,
output_formats=[HumanOutputFormat(sys.stdout, max_length=90), WandBOutputFormat()],
)
model = PPO("MlpPolicy", env, verbose=2)
# Set custom logger
model.set_logger(loggers)
model.learn(total_timesteps=10_000, log_interval=1, callback=LoggerCallback(sinergym_logger=True))Please do let me know if this helps with this issue. Let me know if I should make a PR or if I could help in another way. Thanks!
AlejandroCN7
Feature 🚀
The integration of the "Weights & Biases" (wandb) library in Sinergym.
Motivation
This library may be specially interesting, as it integrates both the functionalities of TensorBoard and MLflow.
As explained by @jajimer:
Solution
Deeply study how wandb could be integrated, even considering if it can completely replace TensorBoard and MLflow or not.
Checklist
:pencil: Please, don't forget to include more labels besides
Feature requestif it is necessary.