Env manager is closed after wandb logger is set

[webbery]

• [ ] I have marked all applicable categories:
  • [x] exception-raising bug
  • [ ] RL algorithm bug
  • [ ] system worker bug
  • [ ] system utils bug
  • [ ] code design/refactor
  • [ ] documentation request
  • [ ] new feature request
• [x] I have visited the readme and doc
• [x] I have searched through the issue tracker and pr tracker
• [x] I have mentioned version numbers, operating system and environment, where applicable:

  import ding, torch, sys
  print(ding.__version__, torch.__version__, sys.version, sys.platform)

Here is my code snap:

def main():
    filename = '{}/log.txt'.format(main_config.exp_name)
    logging.getLogger(with_files=[filename]).setLevel(logging.INFO)
    cfg = compile_config(main_config, create_cfg=create_config, auto=True)

    logger=dict(record_path = './mario_record',
        gradient_logger=True,
        plot_logger= True,
        action_logger = 'action probability',
        )

    wandb.init(
        # Set the project where this run will be logged
        project='SuperMarioBros-1-1-v0',
        # We pass a run name (otherwise it’ll be randomly assigned, like sunshine-lollypop-10)
        name=f"dqn",
        # Track hyperparameters and run metadata
        config=cfg
    )
    with task.start(async_mode=False, ctx=OnlineRLContext()):
        collector_env_num, evaluator_env_num = cfg.env.collector_env_num, cfg.env.evaluator_env_num
        collector_env = SubprocessEnvManagerV2(
            env_fn=[wrapped_mario_env for _ in range(collector_env_num)], cfg=cfg.env.manager
        )
        evaluator_env = SubprocessEnvManagerV2(
            env_fn=[wrapped_mario_env for _ in range(evaluator_env_num)], cfg=cfg.env.manager
        )
        evaluator_env.enable_save_replay(cfg.env.replay_path)

        set_pkg_seed(cfg.seed, use_cuda=cfg.policy.cuda)

        model = DQN(**cfg.policy.model)
        buffer_ = DequeBuffer(size=cfg.policy.other.replay_buffer.replay_buffer_size)
        policy = DQNPolicy(cfg.policy, model=model)

        task.use(wandb_online_logger(logger, evaluator_env, model))

Report error is:

/opt/conda/lib/python3.7/site-packages/ding/envs/env_manager/base_env_manager.py in _check_closed(self)
    220             Check whether the env manager is closed. Will be called in ``__getattr__`` and ``step``.
    221         """
--> 222         assert not self._closed, "env manager is closed, please use the alive env manager"
    223 
    224     def launch(self, reset_param: Optional[Dict] = None) -> None:

AssertionError: env manager is closed, please use the alive env manager

Do I write code correctly? The Guild is not clearly.

[zjowowen]

The wandb logger is being maintained and the API has been changed a lot so far. I reccomend to use a developing branch: https://github.com/zjowowen/DI-engine/tree/new-pipeline-agent

The error occurs due to a closed env manager being used inside wandb_online_logger. The env manger would be lauched if evaluator middleware is used before using wandb_online_logger.

You can turn to this example for help: https://github.com/zjowowen/DI-engine/blob/new-pipeline-agent/ding/bonus/dqn.py#L126

[webbery]

Thanks.

I have to change my code from jupyter to local python.