Memory Leakage while training with stable-baselines

import gym_gvgai
import os
from stable_baselines.deepq.policies import CnnPolicy
from stable_baselines import DQN  # test arbitrary agent
from stable_baselines.results_plotter import load_results, ts2xy
from stable_baselines.bench import Monitor
import numpy as np

best_mean_reward, n_steps = -np.inf, 0

log_dir = "./weights/"

env = gym_gvgai.make('gvgai-golddigger-lvl0-v0')
env = Monitor(env, log_dir, allow_early_resets=True)

model = DQN(CnnPolicy, env, verbose=1,prioritized_replay=True, buffer_size= 1000000, exploration_final_eps=0.1, train_freq=4)

model.learn(total_timesteps=int(100000))
model.save("weights/new_"+"100000"+".pkl")

I have tried it in Google Colab, GCP Compute Engine and my PC, with same results.

I train it with DQN from stable baselines and the RAM usage keeps on increasing steadily. It went on to start from 3.1GB in the beginning to 13GB after 50,000 timesteps. And it stays that way(13GB usage), even if I close the bash terminal. The only way to free the RAM is to restart the system.

I tried running this code:

import gym

from stable_baselines.common.policies import MlpPolicy
from stable_baselines.common.vec_env import SubprocVecEnv
from stable_baselines import A2C

n_cpu = 8
env = SubprocVecEnv([lambda: gym.make('CartPole-v1') for _ in range(n_cpu)])

model = A2C(MlpPolicy, env)
model.learn(total_timesteps=int(1e10))

It works perfectly, no increase in RAM.

SUSTechGameAI / GVGAI_GYM

Memory Leakage while training with stable-baselines #4