Farama-Foundation / SuperSuit

A collection of wrappers for Gymnasium and PettingZoo environments (being merged into gymnasium.wrappers and pettingzoo.wrappers
Other
441 stars 56 forks source link

reset() got an unexpected keyword argument 'seed' #201

Closed frostyduck closed 1 year ago

frostyduck commented 1 year ago

I have one problem. When I run the bellow code

import multiprocessing
import sys
from pettingzoo.test import parallel_api_test
from citylearn import GridLearn
from citylearn import MyEnv
from pathlib import Path
from stable_baselines3.ppo import MlpPolicy
from stable_baselines3 import PPO
import gym
import numpy as np
import supersuit as ss
from copy import deepcopy
import time
import os
import random
import time
random.seed(12)
np.random.seed(12)

model_name = "test"

tic = time.time()

climate_zone = 1
data_path = Path("../citylearn/data/Climate_Zone_"+str(climate_zone))
buildings_states_actions = '../citylearn/buildings_state_action_space.json'

config = {
    "model_name":model_name,
    "data_path":data_path,
    "climate_zone":climate_zone,
    "buildings_states_actions_file":buildings_states_actions,
    "hourly_timesteps":4,
    "percent_rl":0.5,
    "nclusters":1,
    "max_num_houses":None
}

grid = GridLearn(**config)

env = MyEnv(grid) #for _ in range(config['nclusters'])
env.grid = grid
env.initialize_rbc_agents()

print('creating pettingzoo env...')
env = ss.pettingzoo_env_to_vec_env_v1(env)

print('stacking vec env...')
env = ss.concat_vec_envs_v1(env, 1, num_cpus=1, base_class='stable_baselines3')
grid.normalize_reward()
model = PPO(MlpPolicy, env, ent_coef=0.1, learning_rate=0.001, n_epochs=30)
env.reset()
print('==============')
model.learn(4*4*8759, verbose=2)
print('==============')

if not os.path.exists(f'models/{model_name}'):
    os.makedirs(f'models/{model_name}')
os.chdir(f'models/{model_name}')
model.save(f"model")
os.chdir('../..')

toc = time.time()
print(toc-tic)

I have got the following error

Traceback (most recent call last): File "/home/isemadmin/GridLearn-master/tests/pezTest.py", line 66, in env.reset() File "/home/isemadmin/CityLearn-master/city/lib/python3.7/site-packages/supersuit/vector/sb3_vector_wrapper.py", line 14, in reset return self.venv.reset() File "/home/isemadmin/CityLearn-master/city/lib/python3.7/site-packages/supersuit/vector/concat_vec_env.py", line 43, in reset for vec_env in self.vec_envs File "/home/isemadmin/CityLearn-master/city/lib/python3.7/site-packages/supersuit/vector/concat_vec_env.py", line 43, in for vec_env in self.vec_envs File "/home/isemadmin/CityLearn-master/city/lib/python3.7/site-packages/supersuit/vector/markov_vector_wrapper.py", line 56, in reset _observations = self.par_env.reset(seed=seed, options=options) TypeError: reset() got an unexpected keyword argument 'seed']

I have read about this problem of interactions of gym with different wrappers, but I dont understand how I can fix it. I have been trying to install different versions of gym (from 0.14), including the latest (0.26.2), but it did not fix the problem. I have got this error in Windows and Linux for different version of Python (from 3.7 to 3.9) and for conda and virtual envs. Please help me!

My setting the following:

PettingZoo==1.22.0
SuperSuit==3.7.0
stable_baselines3==1.6.2
gym==0.26.2
WillDudley commented 1 year ago

From https://github.com/intelligent-environments-lab/CityLearn/blob/bb3398d76b0e7f3b869f6a41b796b0a178b4aef4/citylearn.py#L653 it seems like CityLearn uses a slightly older seeding convention. It appears to use gym==0.14.0. Here, you call the env.seed() method

I'm unfamiliar with the CityLearn environment (Farama doesn't maintain or develop it), so we are unlikely to be able to assist in detail. However, I wouldn't recommend mismatching library versions like you've done - this will cause errors

Please feel free to reopen this issue if I can assist further