pagand / Eco-Light

Environment friendly traffic signal control with deep reinforcement learning
23 stars 0 forks source link

Issue with the output problem #2

Open sIHURs opened 9 months ago

sIHURs commented 9 months ago

Hi, I am a student who has just started learning reinforcement learning, and I am very grateful for the work you have provided. I am currently trying to run the code, but I have encountered some problems. When running the Q-learning and sarsa algorithm, the output CSV file shows that after 5000-6000 steps, all values become 0, and I am not sure what the problem is.

sarsa: sarsa

ql: ql

where i changed in the code: (in the traffic_signal.py)

traci.trafficlight.getCompleteRedYellowGreenDefinition -> traci.trafficlight.getAllProgramLogics image

traci.trafficlight.setCompleteRedYellowGreenDefinition -> traci.trafficlight.setProgramLogic image

I tried different reward functions, and all other parameters set to the defaults just like in repo, but the results were the same, all info values after 5000-6000 steps will become 0. If you could find the time to reply, I would be very grateful :D

Thanks

wjt0716-DJTU commented 9 months ago

Hello, I have also noticed this issue. When observing the sumo running interface, there are no vehicles starting after 5000-6000 steps. When opening the route file, it was found that there was no defined traffic flow after 5000 steps. I think this is the cause of the problem.

你好,我也发现了这个问题,观察sumo运行界面时,5000-6000步之后就开始没有车辆。打开route文件时发现5000步后没有定义车流。我认为这就是导致问题的原因。我也刚刚开始学习关于交通信号控制的强化学习,看你ID应该是中国人吧?也许我们可以加个联系方式以便相互交流学习。感谢。

sIHURs commented 9 months ago

Hello, I have also noticed this issue. When observing the sumo running interface, there are no vehicles starting after 5000-6000 steps. When opening the route file, it was found that there was no defined traffic flow after 5000 steps. I think this is the cause of the problem.

你好,我也发现了这个问题,观察sumo运行界面时,5000-6000步之后就开始没有车辆。打开route文件时发现5000步后没有定义车流。我认为这就是导致问题的原因。我也刚刚开始学习关于交通信号控制的强化学习,看你ID应该是中国人吧?也许我们可以加个联系方式以便相互交流学习。感谢。

yeah :D my email: yifanwang812@gmail.com, you can send me your wechat or whatsapp