search

intelligent-environments-lab / CityLearn

Official reinforcement learning environment for demand response and load shaping
MIT License
462 stars 167 forks source link

TypeError for when running model.learn(episodes=1, deterministic_finish=True) #56

Closed lijiayi9712 closed 1 year ago

lijiayi9712 commented 1 year ago

Issue Description

When I am trying to reproduce the results from Quickstart section - Decentralized-Independent SAC, I got TypeError with detailed message:

"--------------------------------------------------------------------------- TypeError Traceback (most recent call last)

in 8 env = CityLearnEnv(dataset_name, central_agent=False, simulation_end_time_step=1000) 9 model = RLAgent(env) ---> 10 model.learn(episodes=1, deterministic_finish=True) 11 12 # print cost functions at the end of episode ~/opt/anaconda3/lib/python3.8/site-packages/citylearn/agents/base.py in learn(self, episodes, keep_env_history, env_history_directory, deterministic, deterministic_finish, logging_level) 139 140 while not self.env.done: --> 141 actions = self.predict(observations, deterministic=deterministic) 142 143 # apply actions to citylearn_env ~/opt/anaconda3/lib/python3.8/site-packages/citylearn/agents/sac.py in predict(self, observations, deterministic) 183 184 if self.time_step > self.end_exploration_time_step or deterministic: --> 185 actions = self.get_post_exploration_prediction(observations, deterministic) 186 187 else: ~/opt/anaconda3/lib/python3.8/site-packages/citylearn/agents/sac.py in get_post_exploration_prediction(self, observations, deterministic) 199 for i, o in enumerate(observations): 200 o = self.get_encoded_observations(i, o) --> 201 o = self.get_normalized_observations(i, o) 202 o = torch.FloatTensor(o).unsqueeze(0).to(self.device) 203 result = self.policy_net[i].sample(o) ~/opt/anaconda3/lib/python3.8/site-packages/citylearn/agents/sac.py in get_normalized_observations(self, index, observations) 224 def get_normalized_observations(self, index: int, observations: List[float]) -> npt.NDArray[np.float64]: 225 # try: --> 226 return (np.array(observations, dtype = float) - self.norm_mean[index])/self.norm_std[index] 227 # except: 228 # # print("unable to get normalized observations") TypeError: unsupported operand type(s) for -: 'float' and 'NoneType'" ## Expected Behavior Function get_normalized_observations is supposed to normalize the observations. ## Actual Behavior I printed norm_mean and norm_std within get_normalized_observations from sac.py and found that they are all None, from the initialization. ## Steps to Reproduce I just copied the code from Quickstart section: from citylearn.citylearn import CityLearnEnv from citylearn.agents.sac import SAC as RLAgent dataset_name = 'citylearn_challenge_2022_phase_1' env = CityLearnEnv(dataset_name, central_agent=False, simulation_end_time_step=1000) model = RLAgent(env) model.learn(episodes=2, deterministic_finish=True) ## Environment - CityLearn version: 1.8 - Operating System: OS - Python version: 3 ## Possible Solution If you have any ideas for how to fix the issue, please describe them here. ## Additional Notes Please provide any additional information that may be helpful in resolving this issue.
allenjeffreywu commented 1 year ago

I am unable to reproduce the issue. Were you using CityLearn 1.8 with the current notebook - or the notebook for 1.8 on the current version of CityLearn? Also, what specific version of Python are you using?

allenjeffreywu commented 1 year ago

I was able to recreate the issue, and the most recent commit fixes the notebook. 89783e099b6f8b95d7a7a5173e80444ddda506db

lijiayi9712 commented 1 year ago

Hi Allen,

Thank you so much for getting back to me! I am gonna try this soon!

Best, Jiayi ᐧ

On Thu, Jun 1, 2023 at 8:59 AM Allen Wu @.***> wrote:

I was able to recreate the issue, and the most recent commit fixes the notebook. 89783e0 https://github.com/intelligent-environments-lab/CityLearn/commit/89783e099b6f8b95d7a7a5173e80444ddda506db

— Reply to this email directly, view it on GitHub https://github.com/intelligent-environments-lab/CityLearn/issues/56#issuecomment-1572322025, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGAVNRPBRX7DBAPCLJQO4ZTXJC35HANCNFSM6AAAAAAYSDZ4T4 . You are receiving this because you authored the thread.Message ID: @.***>

lijiayi9712 commented 1 year ago

Hi Allen,

Thanks for your fix! It worked perfectly right now!

I am currently trying to figure out how to access the planned actions and states of each building from the environment. Do you have any suggestions on that? Thanks!

Best, Jiayi ᐧ

On Thu, Jun 1, 2023 at 2:07 PM Jiayi Li @.***> wrote:

Hi Allen,

Thank you so much for getting back to me! I am gonna try this soon!

Best, Jiayi ᐧ

On Thu, Jun 1, 2023 at 8:59 AM Allen Wu @.***> wrote:

I was able to recreate the issue, and the most recent commit fixes the notebook. 89783e0 https://github.com/intelligent-environments-lab/CityLearn/commit/89783e099b6f8b95d7a7a5173e80444ddda506db

— Reply to this email directly, view it on GitHub https://github.com/intelligent-environments-lab/CityLearn/issues/56#issuecomment-1572322025, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGAVNRPBRX7DBAPCLJQO4ZTXJC35HANCNFSM6AAAAAAYSDZ4T4 . You are receiving this because you authored the thread.Message ID: @.***>