search

flow-project / flow

Computational framework for reinforcement learning in traffic control
MIT License
1.07k stars 376 forks source link

How to extend the train/run time? #454

Closed happybyr closed 1 year ago

happybyr commented 5 years ago

I run the tutorial_03, I modify the source code to call the "sumo-gui", but it often stop run in 50.1s and must be restarted, I want to see the further behavior or the car.Fow example, extends the simulation time to 300s. So which parameter should I modify? I just can't find it. I have tryed the 'horizon' and some other parms but didn't work.

AboudyKreidieh commented 5 years ago

hi @happybyr with rllib experiments, two gui's are opened, the first to register the environment and runs one step (since there are 50 warmup seconds the first training step is at 50.1s), and then the second window to open the environment where training occurs. Are you receiving two windows when you run this? if so everything should be working for you

happybyr commented 5 years ago

hi @AboudyKreidieh . I get 2 windows when I run the experiment. Do you mean that increase the warmup_step attribute can extend the train time? I increase it and it works. But when pass more than 200s, the red car disappears. I know that the red car mean the autodrive car. I don't know if it's a common result. Or maybe some other params need to be modify?

happybyr commented 5 years ago

Another question. I get the Flow toolkit from docker pull command. But it seem that many code in the docker image are different with you git repo. Why this happeded?

AboudyKreidieh commented 5 years ago

hi @happybyr sorry for the late reply! For your first issue, can you share with me your Flow git commit number and ray version so that I can try to recreate what you are seeing. As for the second issue, @fywu85 can you look into this?

happybyr commented 5 years ago

I got the Flow from docker pull lucasfischerberkeley/flowdesktop directly. I just want to know how can I extend the train time. Actually I run the rllab example in your tutorial folder. The ideal result should be We witness that, after some training, that the autonomous vehicle learns to dissipate the formation and propagation of "phantom jams" which form when only human driver dynamics is involved., but it run just 50.1s and stop so I can't see the correct result. The result I see is just the autodrive car move very very slowly and the all car slow down. I think maybe it's in the training process and extend the train time can get the correct result. I try to increase the warmup_step but it didn't. I got the explain from the docs These warmup steps are not added as steps into training, and the actions of rl agents during these steps are dictated by sumo. Defaults to zero . So it have no effect to the training process. Well, what I want to know is just how to extend the train time when I use sumo-gui. Which attribute should I modify?

fywu85 commented 5 years ago

Hi @happybyr I apologize. The docker image you are using is very outdated. Can you run a git pull to update your flow inside the docker?

Meanwhile, you mentioned that when you change HORIZON=100, the experiment time does not increase. Theoretically, increasing HORIZON should work. Have you restarted your jupyter notebook kernel after the change? It is likely that the change you made did not propagate to rllib due to the way jupyter notebook works. LMK if this does not help.

happybyr commented 5 years ago

ok, I'll try. But you'd better update your docker image.

happybyr commented 5 years ago

@fywu85 I tryed it. But there are many library incompatible. Bad experience!

CYBruce commented 5 years ago

@AboudyKreidieh @fywu85 I think above question can be solved by increasing the HORIZON. However, I have another question about visualizer_rllib.py. I have trained a bottleneck environment and I want to visualize it. But it always stops at 771s. It gave the information

Reason: TraCI requested termination

Then I checked the code, found this in visualizer_rllib.py.

# lower the horizon if testing
if args.horizon:
config['horizon'] = args.horizon
env_params.horizon = args.horizon

So I change 'config['horizon']' and 'env_params.horizon' to the horizon I want. But it doest work. Did I do something wrong?

fywu85 commented 5 years ago

@CYBruce Can you elaborate more why it does not work? I not quite sure how to handle this but if I were you I would try replacing the following line https://github.com/flow-project/flow/blob/4a3731c876fbb305133f4a2826354c8677a9399c/flow/visualize/visualizer_rllib.py#L198 with something like for _ in range(custom_horizon):.

quangduygtvt commented 4 years ago

Hi @happybyr. I saw you can increase the simulation time to 200s. Can you help me understand how to get 2 windows when I run the experiment and how to increase the simulation time by warmup_step? Thank you so much.