search

CityBrainChallenge / KDDCup2021-CityBrainChallenge-starter-kit

77 stars 40 forks source link

question about action_interval #44

Open KK666-AI opened 3 years ago

KK666-AI commented 3 years ago

Dear author,

I notice that in the demo DQN example, there is a hyperparameter action_interval, its default value is 2. If that, it will break up the assumption of Markov Decision Process (MDP), i.e., the next state for each action is not the exact next state. Also, what's the suggested action_interval will be good enough for the specific problem?

Kanstarry9T commented 3 years ago

action_interval=2 means that each 2*10 = 20 seconds, the agent makes a decision, you can try different integer action_interval values (for example, 1, 2,..), but please be notified the minimum action_interval = 1 (make a decision every 10 seconds).

KK666-AI commented 3 years ago

thanks, i see. according to the setting, which action_interval will be suitable for the problem? Any suggestions based on the best of your knowledge?