Running repo

[alaamohamed2]

Hi ,Hope you are well Hi , I was trying to run this repo but I can’t , can you please tell me how to make testing after training and validation? What Carla version fits this repo? I have Carla 0.9.11 ,does it suit your model? Can I use CPU instead of GPU? can I run this repo on windows? Sorry for making it long . Thanks in advance

[kochlisGit]

Hi,

• First of all, Carla 0.9.11 is fine.
• Yes, If You have a Tensorflow CPU version, You can run it on CPU
• In order to run the agent, You also need tf-agents installed. You can install it from here: https://pypi.org/project/tf-agents/

In order to run this repository, You need place the "code" directory inside "PythonAPI" directory of carla. Then:

1. Run agent/straight_lane_agent_c51_training.py to train the agent.
2. After the training is completed, run agent/straight_lane_agent_c51_validation.py

Unfortunately, I cannot upload the weights of my trained model, since it exceeds the size limit.

If you have any issues, please contact me.

[alaamohamed2]

Thanks a lot for your help ,I have a question does your project follow traffic light and traffic sign rules? thanks in advance

[kochlisGit]

Traffic lights 7 signs are enabled only for NPC Vehicles to obey them. NPC Vehicles work in auto-pilot mode, so they are expected to stop when required. My agent does not obey to traffic lights. However, when there other vehicles in front of him, waiting for the green light, my agent will wait as well.

[alaamohamed2]

Thanks a lot for your help🙏I have made 10000 iterations but still the car made a collision with a front car ,how many iterations should I make so car be able to move just like videos in your demo Thanks in advance

[kochlisGit]

10000 Iterations are a bit low. It took me 200000 iterations to train the vehicle...

However, as I mention in project's readme, there are things You can do to improve the training performance.

1. Train the agent on an empty environment. This way He will learn to drive as fast as possible. Store those weights.
2. Use the weights of the previous training to initialize your agent. Now, the agent already knows that he needs to drive fast! This means that the agent will collide many times at the beginning of the training, which means He will learn faster to keep a distance to front vehicles.

The above process is called Transfer Learning.

Another thing You could try is create a better QNetwork. Currently, my network uses a single Convolutional Layer only. You could try adding more layers.

[alaamohamed2]

Can you tell me the steps to do transfer learning I mean how to use the weight of previous training to initialize current training , what should I do to be able to do transfer learning How many episodes should I make in previous training to initialize current training? Thanks in advance

[kochlisGit]

Open the file "_straight_lane_agent_c51training.py" and check the lines 161-169:

# 7. Building Policy Saver & Checkpointer (Training Saver).
checkpoint_dir = 'checkpoint/'
train_checkpointer = Checkpointer(
    ckpt_dir=checkpoint_dir,
    max_to_keep=1,
    agent=agent,
    policy=agent.policy
)
train_checkpointer.initialize_or_restore()

The train_checkpointer calls the _initialize_orrestore() method, which checks to see If there are any previous checkpoints inside the checkpoint directory. So, in order to do transfer learning, just temporarily remove any cars that are spawned during the training. This can be done by disabling the _trafficmanager in the environment/simulation.py file. Then, train the agent for about 10000-20000 Iterations, until the agent learns to drive very fast.

Later, You can enable the _trafficmanager and run the _straight_lane_agent_c51training.py again. The weights of the previous training should automatically be loaded in the agent.

!! Important !! This agent was trained with an NVIDIA GTX980. If you have more powerful GPU, increasing the parameters & layers of the Categorical Q Network will result in faster training, which means less iterations for your training.

[alaamohamed2]

Thanks a lot for your help , I have a question can you tell me how to disable traffic manager? Thanks in advance

[kochlisGit]

Just remove the lines associated with the traffic manager in the environment/simulation.py in order to remove the vehicles from the scene.

[alaamohamed2]

Thanks a lot for your help , If I create spawns like pedestrians , and I trained model with pedestrians ,can car with training avoid collision with pedestrians? Thanks in advance

[kochlisGit]

The agent is currenly being trained on avoiding collision with vehicles.

Of course You are welcome to expand and improve the agent by addining pedestrians to the environment. In order to do that, You need to add 3 steps:

Step 1.

1. Adding a method to the traffic manager class that i Wrote, that spawns & destroys pedestrians.
2. Call this method from the simulation class at the reset of each episode, along with the vehicles.

For example, the vehicles are spawned in the simulation.py file in the line: traffic_manager.spawn_vehicles() at the reset() method. You could add a method (e.g. spawn_pedestrians()) in the traffic manager that does the same with pedestrians.

Note: There is an example in the carla directory that already shows you how to add pedestrians. The file is inside the examples directory in the PythonAPI folder and is called: spawn_npc.py

Step 2.

Also You need to check the segmentation color of the pedestrians: https://carla.readthedocs.io/en/latest/ref_sensors/#semantic-segmentation-camera

I think it is a red color. Then, You need to modify the simulation/image_data_utils.py, in order to add the pedestrians in the observation as well.

Step 3. Currently, the agent's camera zoom's a lot in the front area of the car. The camera's horizontal field of view in the simulation.py file is set to 10. This is because the car is focusing on front vehicles, instead of incoming vehicles/pedestrians. You might want to change that to higher field of view, in order to be able to see incoming pedestrians from the side of the vehicle.

[alaamohamed2]

Hi thanks a lot for your help Can you help me to make car follow traffic light rules please ? I know that I can get status of traffic light from Carla directly but I can’t integrate it with your model can you tell me please how to do it ? Thanks in advance

[kochlisGit]

Hi,

Carla's dev team has already developed a rule-based agent that obeys the traffic lights. You can find the code of the agent inside the file: PythonApi/carla/agents/navigation/basic_agent.py

[Prof-Os]

Hi, can you share with us libraries versions that you have used please

[kochlisGit]

Sure, I will add a documentation today