Problem with carla_env.py

[heraldiclily]

Congratulation for this work ! I'm just beginner in the field of reinforcement learning and I have some basic questions. I'm trying to implement the "a2c_gym.py" function in the Carla environment on windows. I had a problem when calling carla_env.py due to "os.setsid" and "os.getpsid" specific commands to Linux. I would like to know how can I replace them for windows? What is exactly their role in the process? Another request please, how is the model of RL recorded after training? For example, in a CNN case, the output is a .h5 file containing resulting weights. We can use it therefore for further training or test. In the case of RL implemented, how do we get the model for further use?

[praveen-palanisamy]

Hi @YasminaJaafra ,

In the code, to handle multiple instances of the CARLA environment os.getpgid is used and os.setsid is used to create a new session (for demonizing) when launching a new CARLA server process.

These are available in Unix and most likely not in Windows. Although there are ways for you to get it to work on Windows, there might be incompatibility with other components (for example PyTorch / CARLA simulator themselves may have issues running well on windows etc) as well. If you are planning to spend time working on machine learning projects like these, it may be a good idea to use Ubuntu/OS X. If you want to get it working on windows, you can replace the os.getpgid with os.getpid calls which is available on Windows as well and you can get away without using os.setsid for the preexec_fn when using Popen.

Regarding your other question, for the RL agent algorithms in this repository, the model weights are stored to disk as PyTorch models, which also contain some meta-data (like the mean episode reward, best reward, number of steps trained etc) in addition to the model weights. You can find some pre-trained RL model weights in the "trained_models" directory for some chapters. For example: this directory has trained A3C / A2C models for the CARLA environment and a few other environments.

Hope that helps.

[heraldiclily]

Thank you @praveen-palanisamy for your promt answer.

I replaced os.getpgid with os.getpid and it seems working OK.

Regarding the stored model weights, while operating the RL training, I noted that under the path ch8/trained_models, there are two files (A2C_Carla-v0.ptm and A2C_Carla-v0.ptm.agent_params) which update date and size (13 mo and 1 ko) never change. Is the RL saving its new weights only when it produces better rewards?

[praveen-palanisamy]

Glad to know that worked for you on Windows!

As you may have figured, the A2C_Carla-v0.ptm is the actual model weights for the Carla-v0 environment trained using A2C. The A2C_Carla-v0.ptm.agent_params file contains the parameters used by the A2C agent.

You are right. The model stored on the disk will be updated only when the performance of the agent is better than the previously saved model.

[heraldiclily]

Thank you @praveen-palanisamy for your availability.

I have another request regarding Carla environment:

I need to control environment while training, i.e. modify settings from one episode to another. For example, I will run 400 episodes. After each 10 successive episodes, I will change the degree of environment complexity : number of pedestrians and vehicles, weather id... How could I operate this dynamic setting? I reviewed your book Hands-On Intelligent Agents with OpenAI Gym but I wasn't able to figure out a solution. Is it possible to make it randomly?

[praveen-palanisamy]

Yes. You could change the environment settings (number of vehicles/pedestrians etc) for every new episode if you want by creating a new environment instance directly using the CarlaEnv class. This will be useful for curriculum learning like setups.

For example, you can create new environment instances using: env = CarlaEnv(env_config) where env_config is a dictionary similar to the ENV_CONFIG dictionary.

You can change the complexity of the environment by changing the scenarios parameter. This wiki page explains how to create custom scenarios in CARLA gym which can be used to change the number of vehicles, pedestrians etc to increase/change the complexity of the driving environment.

Once you have created the driving scenarios you want (using this wiki ), you can choose to pick the complexity/scenarios at random after every n episodes. To pick the driving scenario/ environment at random, you can pass a list of Scenarios to the "scenarios" field in the config dictionary that you pass to CarlaEnv(config). For example, if you want to randomly choose between Lane Keeping, Turning along Curves, Dynamic obstacles etc, you can use something like the following code:

...
env_config = {
    "discrete_actions": True,
    "use_image_only_observations": True,  # Exclude high-level planner inputs & goal info from the observations
    "server_map": "/Game/Maps/" + city,
    "scenarios": [scenario_config["Lane_Keep_Town2"], scenario_config["Curve1_Town2"], 
                scenario_config["Nav_Town2"]],
    "framestack": 2,  # note: only [1, 2] currently supported
    "enable_planner": True,
    "use_depth_camera": False,
    "early_terminate_on_collision": True,
    "verbose": False,
    "render" : True,  # Render to display if true
    "render_x_res": 800,
    "render_y_res": 600,
    "x_res": 80,
    "y_res": 80,
    "seed": 1
}
env = CarlaEnv(env_config)
...

[praveen-palanisamy]

@YasminaJaafra : Did the above response answer your question?

[heraldiclily]

@praveen-palanisamy

It's working fine, thank you for the assistance ! Sorry for the answer delay.

Best regards

[praveen-palanisamy]

Great to hear that you got it working! Glad to be of assistance.

---

From: YasminaJaafra notifications@github.com Sent: Thursday, January 24, 2019 4:55:42 PM To: PacktPublishing/Hands-On-Intelligent-Agents-with-OpenAI-Gym Cc: Praveen Palanisamy; Mention Subject: Re: [PacktPublishing/Hands-On-Intelligent-Agents-with-OpenAI-Gym] Problem with carla_env.py (#13)

@praveen-palanisamyhttps://github.com/praveen-palanisamy

It's working fine, thank you for the assistance ! Sorry for the answer delay.

Best regards

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/PacktPublishing/Hands-On-Intelligent-Agents-with-OpenAI-Gym/issues/13#issuecomment-457372581, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AEjKskTWXcQLUnDcuoQrrnA--muwgVw7ks5vGivegaJpZM4ZGK__.