search

intelligent-environments-lab / CityLearn

Official reinforcement learning environment for demand response and load shaping
MIT License
462 stars 167 forks source link

[BUG] stable-baselines3 version incompatible with gym environment #63

Closed Xtian-hub closed 1 year ago

Xtian-hub commented 1 year ago

Issue Description

ValueError Traceback (most recent call last) ~\AppData\Local\Temp\ipykernel_23588\1249709747.py in 8 env = StableBaselines3Wrapper(env) 9 model = SAC('MlpPolicy', env) ---> 10 model.learn(total_timesteps=env.time_steps*2) 11 12 # evaluate

e:\Anaconda\envs\pc\lib\site-packages\stable_baselines3\sac\sac.py in learn(self, total_timesteps, callback, log_interval, tb_log_name, reset_num_timesteps, progress_bar) 311 tb_log_name=tb_log_name, 312 reset_num_timesteps=reset_num_timesteps, --> 313 progress_bar=progress_bar, 314 ) 315

e:\Anaconda\envs\pc\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py in learn(self, total_timesteps, callback, log_interval, tb_log_name, reset_num_timesteps, progress_bar) 304 reset_num_timesteps, 305 tb_log_name, --> 306 progress_bar, 307 ) 308

e:\Anaconda\envs\pc\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py in _setup_learn(self, total_timesteps, callback, reset_num_timesteps, tb_log_name, progress_bar) 287 reset_num_timesteps, 288 tb_log_name, --> 289 progress_bar, 290 ) 291

e:\Anaconda\envs\pc\lib\site-packages\stable_baselines3\common\base_class.py in _setup_learn(self, total_timesteps, callback, reset_num_timesteps, tb_log_name, progress_bar) 422 assert self.env is not None 423 # pytype: disable=annotation-type-mismatch --> 424 self._last_obs = self.env.reset() # type: ignore[assignment] 425 # pytype: enable=annotation-type-mismatch 426 self._last_episode_starts = np.ones((self.env.num_envs,), dtype=bool)

e:\Anaconda\envs\pc\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py in reset(self) 74 def reset(self) -> VecEnvObs: 75 for env_idx in range(self.num_envs): ---> 76 obs, self.reset_infos[env_idx] = self.envs[env_idx].reset(seed=self._seeds[env_idx]) 77 self._save_obs(env_idx, obs) 78 # Seeds are only used once

e:\Anaconda\envs\pc\lib\site-packages\stable_baselines3\common\monitor.py in reset(self, kwargs) 81 raise ValueError(f"Expected you to pass keyword argument {key} into reset") 82 self.current_reset_info[key] = value ---> 83 return self.env.reset(kwargs) 84 85 def step(self, action: ActType) -> Tuple[ObsType, SupportsFloat, bool, bool, Dict[str, Any]]:

e:\Anaconda\envs\pc\lib\site-packages\shimmy\openai_gym_compatibility.py in reset(self, seed, options) 239 ) 240 --> 241 obs = self.gym_env.reset() 242 243 if self.render_mode == "human":

e:\Anaconda\envs\pc\lib\site-packages\gym\core.py in reset(self, kwargs) 321 def reset(self, kwargs) -> Tuple[ObsType, dict]: 322 """Resets the environment with kwargs.""" --> 323 return self.env.reset(**kwargs) 324 325 def render(

e:\Anaconda\envs\pc\lib\site-packages\gym\core.py in reset(self, kwargs) 377 def reset(self, kwargs): 378 """Resets the environment, returning a modified observation using :meth:self.observation.""" --> 379 obs, info = self.env.reset(**kwargs) 380 return self.observation(obs), info 381

e:\Anaconda\envs\pc\lib\site-packages\gym\core.py in reset(self, kwargs) 321 def reset(self, kwargs) -> Tuple[ObsType, dict]: 322 """Resets the environment with kwargs.""" --> 323 return self.env.reset(**kwargs) 324 325 def render(

e:\Anaconda\envs\pc\lib\site-packages\gym\core.py in reset(self, kwargs) 321 def reset(self, kwargs) -> Tuple[ObsType, dict]: 322 """Resets the environment with kwargs.""" --> 323 return self.env.reset(**kwargs) 324 325 def render(

e:\Anaconda\envs\pc\lib\site-packages\gym\core.py in reset(self, kwargs) 377 def reset(self, kwargs): 378 """Resets the environment, returning a modified observation using :meth:self.observation.""" --> 379 obs, info = self.env.reset(**kwargs) 380 return self.observation(obs), info 381

ValueError: not enough values to unpack (expected 2, got 1)

I ran quickstart.ipynb without making any changes and it throws this error, may I ask why?

the site of quickstart.ipynb is https://github.com/intelligent-environments-lab/CityLearn/blob/master/examples/quickstart.ipynb

Expected Behavior

Environment

@kingsleynweye Kingsley Nweye

kingsleynweye commented 1 year ago

@Xtian-hub, Iine 379 in your output was the issue:

--> 379 obs, info = self.env.reset(**kwargs)

The latest stable-baselines3 version supports gymnasium environments and the reset function in those return 2 values unlike gym that CityLearn is built on. I have hard-set stable-baselines3==1.8.0 to avoid this issue.

Xtian-hub commented 1 year ago

sorry, i'm a newbie in python. the issue is about releases of gym and stable-baselines3 from your answer. Am I misunderstanding? i pip install stable-baselines3==1.6.0 gym=0.21.0,but issue still here. Package Version

asttokens 2.2.1 backcall 0.2.0 backports.functools-lru-cache 1.6.5 certifi 2023.7.22 charset-normalizer 3.2.0 CityLearn 2.0b3 cloudpickle 2.2.1 colorama 0.4.6 contourpy 1.1.0 cycler 0.11.0 debugpy 1.5.1 decorator 5.1.1 executing 1.2.0 Farama-Notifications 0.0.4 filelock 3.12.2 fonttools 4.41.1 gym 0.21.0 gym-notices 0.0.8 gymnasium 0.28.1 idna 3.4 importlib-metadata 4.13.0 importlib-resources 6.0.0 ipykernel 6.15.0 ipython 8.14.0 jax-jumpy 1.0.0 jedi 0.18.2 Jinja2 3.1.2 joblib 1.3.1 jupyter_client 8.3.0 jupyter_core 5.3.1 kiwisolver 1.4.4 MarkupSafe 2.1.3 matplotlib 3.7.2 matplotlib-inline 0.1.6 mpmath 1.3.0 nest-asyncio 1.5.6 networkx 3.1 numpy 1.25.1 packaging 23.1 pandas 1.3.5 parso 0.8.3 pickleshare 0.7.5 Pillow 10.0.0 pip 23.1.2 platformdirs 3.9.1 prompt-toolkit 3.0.39 psutil 5.9.0 pure-eval 0.2.2 Pygments 2.15.1 pyparsing 3.0.9 python-dateutil 2.8.2 pytz 2023.3 pywin32 305.1 pyzmq 25.1.0 requests 2.31.0 scikit-learn 1.0.2 scipy 1.11.1 setuptools 63.2.0 Shimmy 1.2.1 simplejson 3.19.1 six 1.16.0 stable-baselines3 1.6.0 stack-data 0.6.2 sympy 1.12 threadpoolctl 3.2.0 torch 2.0.1 torchvision 0.15.2 tornado 6.2 traitlets 5.9.0 typing_extensions 4.7.1 urllib3 2.0.4 wcwidth 0.2.6 wheel 0.38.4 zipp 3.16.2 Note: you may need to restart the kernel to use updated packages.

kingsleynweye commented 1 year ago

@Xtian-hub correct it is about the versions of both packages. As the output says, you need to restart your jupyter notebook kernerl to use the updated package version.

Xtian-hub commented 1 year ago

It work. Thank you for helping me！