search

XingshuaiHuang / DTLight-old

Traffic Signal Control Using Lightweight Transformers: An Offline-to-Online RL Approach (https://arxiv.org/abs/2312.07795)
MIT License
3 stars 0 forks source link

Unable to reproduce results #2

Open PoisonS1K opened 5 months ago

PoisonS1K commented 5 months ago

Hello,

Thank you for your research efforts, your work was very interesting to read, I however have some questions regarding how to replicate the results described in the paper.

Running all default settings with the exception of --map "cologne1" and --behavior_policy "EMP", the resulting model seems to get stuck on a single phase, with the delay time around 20 seconds, because the vehicle flow is stopped. I have also tested on 2lane and had the same results. Is there an issue with the default settings?

Additionally, the yellow phase seems to be broken in the string comparison inside prep_phase, resulting in no yellow phases.

Thank you for your assistance.

XingshuaiHuang commented 5 months ago

Hi,

Thank you for your interest in our project and for taking the time to investigate this issue. I've reviewed the code and tested the model, but I couldn't replicate the problem you described regarding the stuck phase. I've attached a screenshot showing the phase selections, and everything seems to be functioning correctly with the teacher/student and online models.

image image

image

Regarding the SUMO environment settings, I've utilized the same code as the RESCO benchmark without significant modifications. However, if you could provide more details about the issue you're encountering, and perhaps include some screenshots, it would greatly assist in diagnosing the problem more accurately.

Namasivaayam-L commented 5 months ago

Hey really great repo you got here. I came to know this by reading your paper wonderful research... congrats!.. While going through I got a doubt... In the .rou files you have created for the environments.. How do you calculate the maximum number of vehicles that the network can handle? In my personal project.. while using multi-agent environments.. vehicles get stagnated at the end of the simulation in sumo. As a result the waiting time of the system keeps on increasing like a curve. Can you please explain how to prevent this issue and what method you used?

PoisonS1K commented 4 months ago

image (teacher model) This issue happens with both MA2C and MAXPRESSURE config policy, as well as IDQN and EMP behavior policy (the only ones I have tested). It appears that once vehicles reach a certain lane density, the agent stops selecting that phase altogether. In this specific screenshot of the teacher model, it happened around 3200 steps and remained the same action for the remainder of the simulation.

Another example with IDQN behavior_policy + MAXPRESSURE config_policy, occuring around 2700 steps and remaining on the N_S phase until the simulation ends image (distillation model)

image (finetune model)

I am curious what is causing this. Thanks!

XingshuaiHuang commented 4 months ago

image (teacher model) This issue happens with both MA2C and MAXPRESSURE config policy, as well as IDQN and EMP behavior policy (the only ones I have tested). It appears that once vehicles reach a certain lane density, the agent stops selecting that phase altogether. In this specific screenshot of the teacher model, it happened around 3200 steps and remained the same action for the remainder of the simulation.

Another example with IDQN behavior_policy + MAXPRESSURE config_policy, occuring around 2700 steps and remaining on the N_S phase until the simulation ends image (distillation model)

image (finetune model)

I am curious what is causing this. Thanks!

Thank you for pointing this out. Upon running the code from the clean repository, I encountered some issues that were not present in my local repository. It appears that I may have deleted something by mistake when uploading the clean repository. I have since made the necessary corrections and ensured that the datasets are accurate. I retested the repository for all scenarios and reproduced similar results to those shown in my paper. I hope this resolves your concerns.

XingshuaiHuang commented 4 months ago

Hey really great repo you got here. I came to know this by reading your paper wonderful research... congrats!.. While going through I got a doubt... In the .rou files you have created for the environments.. How do you calculate the maximum number of vehicles that the network can handle? In my personal project.. while using multi-agent environments.. vehicles get stagnated at the end of the simulation in sumo. As a result the waiting time of the system keeps on increasing like a curve. Can you please explain how to prevent this issue and what method you used?

Hey, thanks for your interest. I didn't calculate the maximum number of vehicles the network can handle but adjusted some key args in randomTrips.py to generate a proper route file.