intelligent-environments-lab / CityLearn

Official reinforcement learning environment for demand response and load shaping
MIT License
468 stars 171 forks source link

[BUG] Inconsistent update function #100

Closed joshua-cogliati-inl closed 8 months ago

joshua-cogliati-inl commented 9 months ago

Issue Description

citylearn/agents/base.py has this call to update: self.update(observations, actions, rewards, next_observations, done=done) but citylearn/agents/q_learning.py update has this definition: def update(self, observations: List[List[float]], actions: List[List[float]], reward: List[float], next_observations: List[List[float]]) which does not have an done in the update which results in this error:

File ~/.conda/envs/citytest310c/lib/python3.10/site-packages/citylearn/agents/base.py:155, in Agent.learn(self, episodes, deterministic, deterministic_finish, logging_level)
    153 # update
    154 if not deterministic:
--> 155     self.update(observations, actions, rewards, next_observations, done=done)
    156 else:
    157     pass

TypeError: TabularQLearning.update() got an unexpected keyword argument 'done'

if you run examples/citylearn_rlem23_tutorial.ipynb with a current version of citylearn.

Expected Behavior

The example to work with a current version of citylearn.

Actual Behavior

The above error.

Steps to Reproduce

Use the code in examples/citylearn_rlem23_tutorial.ipynb with citylearn 2.1.0

Environment

Possible Solution

Either remove the done=done from base.py, or add back the done parameter removed in 3b562b954c78e432f02ddd8a6f11f824fc5abedd

other information

I realize the example is targeted to version 1.8.0, but I am not sure how you would use TabularQLearning in version 2.1.0 without triggering this issue.

Thanks.

kingsleynweye commented 8 months ago

@joshua-cogliati-inl thanks for spotting the bug. It has been fixed by including done in the TabularQLearning.update() method. Please update to the latest release CityLearn==2.1.1.