How to save wandb Logs offline?

[Saood1810]

Hi

I am using a GPU cluster and I training the PQL algorithm. The issue is the GPU cluster does not have access to the Internet, meaning I need to use Wandb in offline mode so I can save the plots locally and then sync them online after training. How would I do given that I can only enter the Experiment and Project name parameters for Wandb when initialising the algorithm? I tried being creative but nothing seems to work.

In the approach I did below, I initialised and in offline mode, and the code works fine (it doesn't try and make a connection to the Internet to log plots online). But now, after running, I have issues with the log file when I sync it online. So i don't think it is getting logged on properly. Any suggestions?

import mo_gymnasium as mo_gym import numpy as np import random from morl_baselines.multi_policy.pareto_q_learning.pql import PQL import os import wandb

SEEDS = [42,43,44]
env = mo_gym.make("deep-sea-treasure-concave-v0") ref_point = np.array([0, -25])

wandb.init(mode="offline",project="Research Project Logs") for seed in SEEDS:

print(f"Running experiment with seed {seed}")
env.reset(seed=seed)
random.seed(seed)
np.random.seed(seed)
env.reset(seed=seed)

agent = PQL(
    env,
    ref_point,
    gamma=0.9,
    initial_epsilon=1,
    epsilon_decay_steps=10000,
    final_epsilon=0.1,
    seed=seed,
    experiment_name="Pareto Q-Learning in DST",
    log=True,)

pf = agent.train(
    total_timesteps=1000000,
    log_every=100,
    action_eval="hypervolume",
    known_pareto_front=env.pareto_front(gamma=0.99),
    ref_point=ref_point,
    eval_env=env,)
print(pf)

# Execute a policy
target = np.array(pf.pop())
print(f"Tracking {target}")
reward = agent.track_policy(target, env=env)
print(f"Obtained {reward}")
wandb.finish()

[ffelten]

Hey,

I have no experience with wandb offline on a cluster. Sometimes I use it locally with their docker image but I guess that doesn't help for a cluster. You might have more luck on the wandb support instead of here.

[Saood1810]

Oh okay Alright!