Closed Ganesh-mali closed 5 months ago
Hi @Ganesh-mali, there is a related issue to reference for this https://github.com/intelligent-environments-lab/CityLearn/issues/63. You need to downgrade your stable-baselines3 installation, preferably to 1.7.0 or 1.8.0. CityLearn is still a gym environment whereas later stable-baselines3 releases support gymnasium environments.
hey @kingsleynweye thank you for your reply, I tried downgrading to both 1.7.0 as well as 1.8.0. I get new error with that :
Building wheels for collected packages: gym Building wheel for gym (setup.py) ... error error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [477 lines of output]
Let me know , if you need more info.
@Ganesh-mali this error is a recently known gym
bug for version 0.21. I have been able to bypass it in the past by downgrading pip and setuptools before installing CityLearn and stable-baselines3:
pip install pip==21.3.1
pip install setuptools==41.2.0
pip install CityLearn==2.0b4
pip install stable-baselines3==1.7.0
@Ganesh-mali hmmnn I have not seen this before. If I understand correctly, it is the stable-baselines3
installation throwing this new error? Also, looks like you are having issues with anaconda virtual environments. I am not very familiar with using anaconda as I typically use venv
to manage virtual environments.
hey @kingsleynweye , this time I created a virtual environment using Venv and repeated your steps, Again same error, the first three ran successfully and the installation of stable-baseline3 failed again.
Installing collected packages: zipp, gym, importlib-metadata, stable-baselines3
Attempting uninstall: gym
Found existing installation: gym 0.26.2
Uninstalling gym-0.26.2:
Successfully uninstalled gym-0.26.2
DEPRECATION: gym is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
Running setup.py install for gym ... error
error: subprocess-exited-with-error
× Running setup.py install for gym did not run successfully.
│ exit code: 1
╰─> [573 lines of output]
running install
running build
running build_py
creating build
This above message followed some 200 line output then it showed :
, line 2449, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "C:\COURSES_LEARNINGS\Rl_learning\city_learn\city_test\lib\site-packages\setuptools\command\bdist_wininst.py", line 1, in <module>
import distutils.command.bdist_wininst as orig
ModuleNotFoundError: No module named 'distutils.command.bdist_wininst'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
Rolling back uninstall of gym
Moving to c:\courses_learnings\rl_learning\city_learn\city_test\lib\site-packages\gym-0.26.2.dist-info\
from C:\COURSES_LEARNINGS\Rl_learning\city_learn\city_test\Lib\site-packages\~ym-0.26.2.dist-info
Moving to c:\courses_learnings\rl_learning\city_learn\city_test\lib\site-packages\gym\
from C:\COURSES_LEARNINGS\Rl_learning\city_learn\city_test\Lib\site-packages\~ym
error: legacy-install-failure
If you need more info please let me know. Or If you have any other suggestions, I will follow your guidelines I just want to run the code that is on Citylearn documentation.
Hi @kingsleynweye
I have an update, I have installed all for packages as you said, After running the code I am getting this error now :
C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\torch\nn\modules\rnn.py:89: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xe (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:68.)
w_ih = Parameter(torch.empty((gate_size, layer_input_size), **factory_kwargs))
Traceback (most recent call last):
File "C:\COURSES_LEARNINGS\Rl_learning\city_learn\p1-city_learn.py", line 12, in <module>
model.learn(total_timesteps=env.time_steps*20)
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\sac\sac.py", line 299, in learn
return super().learn(
File "C:\Users\\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 323, in learn
total_timesteps, callback = self._setup_learn(
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 305, in _setup_learn
return super()._setup_learn(
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\base_class.py", line 408, in _setup_learn
self._last_obs = self.env.reset() # pytype: disable=annotation-type-mismatch
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 74, in reset
obs = self.envs[env_idx].reset()
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\stable_baselines3\common\monitor.py", line 83, in reset
return self.env.reset(**kwargs)
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 323, in reset
return self.env.reset(**kwargs)
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 379, in reset
obs, info = self.env.reset(**kwargs)
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 323, in reset
return self.env.reset(**kwargs)
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 323, in reset
return self.env.reset(**kwargs)
File "C:\Users\anaconda3\envs\new_city_test_env\lib\site-packages\gym\core.py", line 379, in reset
obs, info = self.env.reset(**kwargs)
ValueError: not enough values to unpack (expected 2, got 1)
Hey @kingsleynweye any update on this ?
This is indeed a bug. The library needs to depend on gymnasium
, not gym
. The OpenAI's gym
package hasn't been maintained in years, and a independent developer took it upon themself to revive it and add features the RL community wanted, and in the process releases it on pypi as gymnasium
to avoid issues.
@Ganesh-mali this part of the error:
obs, info = self.env.reset(**kwargs)
ValueError: not enough values to unpack (expected 2, got 1)
makes me believe the issue is caused by your installation of stable-baselines3 expecting a gymnasium environment where the reset
function returns a tuple of two values. In line with what @H-Park mentioned, CityLearn
is a gym
environment and gym
environments' step function returns a single value, obs
.
To avoid this problem, you need to use a stable-baselines3 version compatible with gym
(I think any stable-baselines3<=1.8.0).
This is indeed a bug. The library needs to depend on
gymnasium
, notgym
. The OpenAI'sgym
package hasn't been maintained in years, and a independent developer took it upon themself to revive it and add features the RL community wanted, and in the process releases it on pypi asgymnasium
to avoid issues.
@H-Park yeah, ideally, we would upgrade CityLearn
to gymnasium
and it is something on the list but not quite a priority yet :( . If this is a fix you will want to see sooner and have to know-how, please feel free to contribute as CityLearn is open source and I think migrating to gymnasium is a very welcomed community contribution through a pull request.
@Ganesh-mali this part of the error:
obs, info = self.env.reset(**kwargs) ValueError: not enough values to unpack (expected 2, got 1)
makes me believe the issue is caused by your installation of stable-baselines3 expecting a gymnasium environment where the
reset
function returns a tuple of two values. In line with what @H-Park mentioned,CityLearn
is agym
environment andgym
environments' step function returns a single value,obs
.To avoid this problem, you need to use a stable-baselines3 version compatible with
gym
(I think any stable-baselines3<=1.8.0).
Forgot to mention that another way I have been able to solve this problem is by installing shimmy that provides a gymnasium
binder.
@kingsleynweye the dataset given in the example code "baeda_3dem" doesn't work, I even checked using the method Datset.get_names(). It printed all available datasets except baeda_3dem. Anyways, it is working with other datasets, I am able to run the code with SB3 (1.7.0), and gym (0.21.0). Thanks for all your suggestions.
@Ganesh-mali, if baeda_3dem
is not getting printed then I suspect you are using a citylearn version that is <=1.8.0
. It was only added after 1.8.0
It prints out for me for later versions.
This is the new issue with running the code, i am not sure...if any of these examples can be run as it is can you please point me where i can run at least one full example on this... Thanks
130 if not deterministic: --> 131 self.update(observations, actions, rewards, next_observations, done=self.env.done) 132 else: 133 pass
TypeError: update() got an unexpected keyword argument 'done'
This is the new issue with running the code, i am not sure...if any of these examples can be run as it is can you please point me where i can run at least one full example on this... Thanks
130 if not deterministic: --> 131 self.update(observations, actions, rewards, next_observations, done=self.env.done) 132 else: 133 pass
TypeError: update() got an unexpected keyword argument 'done'
What examples are you referring to? With the current code, I was able to run all sections of quickstart.ipynb but Stable Baselines3 Reinforcement Learning Algorithms
section in Google Colab. I will push a fix for Stable Baselines3 example soon.
i = 3 m = tql_env.observation_space[0].n n = tql_env.action_space[0].n t = tql_env.time_steps - 1 tql_episodes = mni/t tql_episodes = int(tql_episodes) print('Q-Table dimension:', (m, n)) print('Number of episodes to train:', tql_episodes)
loader = get_loader(max=tql_episodes*t) display(loader)
tql_kwargs = { 'epsilon': 1.0, 'minimum_epsilon': 0.01, 'epsilon_decay': 0.0001, 'learning_rate': 0.005, 'discount_factor': 0.99, }
tql_model = CustomTabularQLearning( env=tql_env, loader=loader, random_seed=RANDOM_SEED, **tqlkwargs ) = tql_model.learn(episodes=tql_episodes)
the last line giving this error. can you please check this once
----------------- CALCULATE NUMBER OF TRAINING EPISODES -----------------
i = 3 m = tql_env.observation_space[0].n n = tql_env.action_space[0].n t = tql_env.time_steps - 1 tql_episodes = m_n_i/t tql_episodes = int(tql_episodes) print('Q-Table dimension:', (m, n)) print('Number of episodes to train:', tql_episodes)
------------------------------- SET LOADER ------------------------------
loader = get_loader(max=tql_episodes*t) display(loader)
----------------------- SET MODEL HYPERPARAMETERS -----------------------
tql_kwargs = { 'epsilon': 1.0, 'minimum_epsilon': 0.01, 'epsilon_decay': 0.0001, 'learning_rate': 0.005, 'discount_factor': 0.99, }
----------------------- INITIALIZE AND TRAIN MODEL ----------------------
tql_model = CustomTabularQLearning( env=tql_env, loader=loader, random_seed=RANDOM_SEED, **tqlkwargs ) = tql_model.learn(episodes=tql_episodes)
the last line giving this error. can you please check this once
Oh I see. The as-provided notebook works fine for me in Colab (https://colab.research.google.com/github/intelligent-environments-lab/CityLearn/blob/master/examples/citylearn_rlem23_tutorial.ipynb). Pleae, can I see what the error message is?
@kingsleynweye
Forgot to mention that another way I have been able to solve this problem is by installing shimmy that provides a gymnasium binder.
Hey,Can you explain specifically how to use [shimmy] to solve this bug? Thanks~
@kingsleynweye
Forgot to mention that another way I have been able to solve this problem is by installing shimmy that provides a gymnasium binder.
Hey,Can you explain specifically how to use [shimmy] to solve this bug? Thanks~
You will want to install it with stable-baselines3. For example:
pip install stable-baselines3==2.1.0
pip install shimmy==0.2.1
Other version numbers for either package might work.
Issue Description I am encountering an error while running the code provided in the official CityLearn documentation. I have not modified a single line of the code, and I'm using the exact code snippet provided.
Expected Behavior I expected the code to run without errors, as it's directly taken from the official CityLearn documentation.
Actual Behavior I am facing the following error: ValueError: not enough values to unpack (expected 2, got 1)
Steps to Reproduce Install CityLearn (version 2.0b4), Stable Baselines 3 (version 2.0.0), and Gym (version 0.26.1). Run the code provided in the official CityLearn documentation:(https://www.citylearn.net/quickstart.html)
Code: from stable_baselines3.sac import SAC from citylearn.citylearn import CityLearnEnv from citylearn.wrappers import NormalizedObservationWrapper, StableBaselines3Wrapper
dataset_name = 'baeda_3dem' env = CityLearnEnv(dataset_name, central_agent=True, simulation_end_time_step=1000) env = NormalizedObservationWrapper(env) env = StableBaselines3Wrapper(env) model = SAC('MlpPolicy', env) model.learn(total_timesteps=env.time_steps*2)
evaluate
observations = env.reset()
while not env.done: actions, = model.predict(observations, deterministic=True) observations, , , = env.step(actions)
kpis = env.evaluate().pivot(index='cost_function', columns='name', values='value') kpis = kpis.dropna(how='all') display(kpis)
Environment CityLearn version: 2.0b4 Operating System: Windows Python version: 3.10
Possible Solution I have tried various solutions and referred to the official documentation, but I am unable to find a compatible version combination that resolves this issue. I'm looking for guidance from the community.
Additional Notes I'm following the instructions exactly as provided in the official documentation, so I'm puzzled as to why I'm encountering this issue. If anyone has experience with these libraries and can provide guidance or suggestions, I would greatly appreciate it.
Thank you for your time and assistance!