question about how to integrate prediction and control model

[SHITIANYU-hue]

I have a question about how to integrate prediction and control models together.

I think using a prediction model, we can measure the sensor input and control input, and then we can predict future energy consumption. For the control problem, we generate an optimal control decision based on the state observation. As we can see, there are two tracks, prediction and control: https://www.aicrowd.com/challenges/neurips-2023-citylearn-challenge.

Based on my knowledge, I think the prediction task is to learn a simulator that could be used for RL training.

When I learned this demo:

from citylearn.agents.rbc import BasicRBC as RBCAgent
from citylearn.citylearn import CityLearnEnv, EvaluationCondition
import citylearn
dataset_name = 'citylearn_challenge_2022_phase_1'
env = CityLearnEnv(dataset_name, central_agent=True, simulation_end_time_step=1000)
model = RBCAgent(env)
model.learn(episodes=4)

## print cost functions at the end of episode
kpis = model.env.evaluate(baseline_condition=EvaluationCondition.WITHOUT_STORAGE_BUT_WITH_PARTIAL_LOAD_AND_PV)
kpis = kpis.pivot(index='cost_function', columns='name', values='value')
kpis = kpis.dropna(how='all')
print(kpis)
print(citylearn.data.DataSet.get_names())

i think it directly trains the model on the dataset, I am a bit confused, should it build a prediction model first, then train the RL agent? I think it is directly trained on the dataset here: https://github.com/intelligent-environments-lab/CityLearn/tree/master/citylearn/data. So, do we really need a prediction model?

[kingsleynweye]

@SHITIANYU-hue thanks for your question. However I am not 100% clear on what it is so I will paraphrase what I understood and answer? From my understanding, you want to know if you need a prediction model to pair with the RL agent to control the buildings? If that is the question, here's my response:

The two tracks in the competition, prediction and control, are independent and you don't need one for the other. Consequently, you don't necessarily need a prediction method to pair with RL for control. However, a prediction model e.g. one that predicts future electricity consumption, solar generation or aggregated electricity consumption might improve your performance as we showed in the MARLISA paper (https://dl.acm.org/doi/10.1145/3408308.3427604).