How to change the width of the bottleneck to single lane in MARL_Bottleneck envirorment?

[dineshresearch]

Hello @QuanyiLi @pengzhenghao

I am working on simulating the below scenario but unable to change the width of the bottleneck

In marl_bottleneck.py by changing the parameters i am able to get this map_config=dict(exit_length=60, bottle_lane_num=2, neck_lane_num=1, neck_length=20, lane_width=3.5, lane_num=1),

Can you please help me for having only single lane at bottleneck so that only 1 vehicle passes through it at a time, what changes i have to do?

[QuanyiLi]

Hi, Dinesh

To achieve this, you have to make a new bottleneck block that only contains one lane but enables bi-direction traffic.

You can look through https://github.com/metadriverse/metadrive/tree/main/metadrive/component/pgblock for how to customize a block. Besides these examples, I would like to give you a hint that you can create two overlapped lanes with 7m width but in different directions. After that, the navigation module of vehicles would still work in this situation.

Quanyi

[dineshresearch]

Hello @QuanyiLi @pengzhenghao

I have built the custom bidirectional bottleneck environment with bidirectional straight road blocks as below. But training the Copo algorithm is leading to inferior results as below.

Attaching modified marl_bottleneck.py, multi_agent_metadrive.py, second_block.py scripts - modified_scripts.zip

Any suggestions from your side to improve success_rate will be helpful. Also how to incorporate the bottleneck information so that vehicles can be aware of the bottleneck?

[dineshresearch]

Hello @QuanyiLi @pengzhenghao

Any updates on this issue?

[pengzhenghao]

Hi @dineshresearch

The environment looks cool! But I am wondering if MetaDrive supports bi-directional lane yet.. I am looking into your code.

Could you please tell what is the detailed behavior of your car?

[dineshresearch]

Yes I have used custom blocks in metadrive to create this custom bidirectional bottleneck. Currently my aim is to train the copo algorithm on this custom bottleneck so that both the vehicles from the opposite ends pass through the bottleneck (as shown in above figure) without any collision and out of road scenarios. But that is not happening as shown in the training plots

my reward config is

===== New Reward Setting =====

out_of_road_penalty=10,
crash_vehicle_penalty=10,
crash_object_penalty=10,
crash_vehicle_cost=1,
crash_object_cost=1,
out_of_road_cost=0,  # Do not count out of road into cost!
neighbours_distance=10, 
respawn = true
num_agents = 2

success rate is < 0.18 and episode_reward_mean is also < 40. Can you please suggest steps to increase the success_rate kindly? thanks in advance.

[pengzhenghao]

I think the first step to diagnose RL performance issue is to look into the environment and see if anything wrong.

That's what my reproduced code generates in latest MetaDrive ( #198 ) and it seems something wrong. I am still looking into the code. Do you have any suggestion on why the env seems broken now?

[pengzhenghao]

If you @dineshresearch have any update, maybe you can push to the branch bidirlane so that we can work on this. I think this bi-directional block is quite interesting and will work it out.

My other comments on RL training:

1. neighbours_distance = 10 seems to be too short? I set it to 40 in CoPO. If the agent can't detect it opponent in other side of the road, then CoPO will just reduce to IPPO I think.
2. To ensure the navigation system is still working (which is tightly coupled to the reward function and observation), I suggest you to run a training with num_agents = 1 with IPPO method. If it can work, then the navigation is working. If it can only achieve 50% success rate, then it is highly probable that the adverse lane is broken.

[dineshresearch]

@pengzhenghao You can replace the scripts init.py, pg_block.py,second_block.py at metadrive/component/blocks, then replace the marl_bottleneck.py at metadrive/envs/marl_envs with the provided files below. This should bring up the custom bidirectional bottleneck. custom_bottle1.zip

RL training:

1. I have tried with neighbours_distance = 40 as well but their is no significant difference in the success_rate.
2. I have trained with num_agents = 1 and obtained good success_rate = 0.84. So I think navigation is working fine.

As you suggested i will create the bidirlane branch and update my changes. Thanks for your support and interest.

[pengzhenghao]

I already push the branch and also update your code to fit into latest MetaDrive. Could you please pull existing bidirlane branch and see if the env satisfies your expectation?

Best regards!

PENG Zhenghao

在 2022年7月8日，13:35，Dinesh Kumar Amara @.***> 写道：

 @pengzhenghao You can replace the scripts init.py, pg_block.py,second_block.py at metadrive/component/blocks, then replace the marl_bottleneck.py at metadrive/envs/marl_envs with the provided files below. This should bring up the custom bidirectional bottleneck. custom_bottle1.zip

As you suggested i will create the bidirlane branch and update my changes. Thanks for your support and interest.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[QuanyiLi]

Hi @dineshresearch,

I updated a totally new environment at /envs/marl_envs/marl_bidirection.py on bidirlane branch, which, in my opinion, satisfies your requirement. Please pull it and test this new environment. Look forward to your feedback!

Best, Quanyi

[dineshresearch]

Sure i will do that and update you

[dineshresearch]

@pengzhenghao @QuanyiLi The changes in screensize that you have made here metadrive/engine/core/engine_core.py is causing the below error. As of now i have fixed it by changing the screen size to (1200, 900) in base_env.py

The environment runs fine as shown below.

But training with Copo i am getting the below error. kindly any help to resolve this error

[pengzhenghao]

Opps, this is a mismatch between CoPO and MetaDrive.

I already fixed it in latest CoPO repo with PR: https://github.com/decisionforce/CoPO/pull/16

Concretely, here is the fix:

https://github.com/decisionforce/CoPO/pull/16/files#diff-701cf460dd97b1364d3f9db4975c0b8d4b9ae36ffa6ed095697e9a85aaf09376R37

[dineshresearch]

@pengzhenghao thanks now the issue is resolved and I am done with the training. @QuanyiLi

1. I have trained this new bidirectional bottleneck environment (bottle_lane_num=2, num_agents =2) with the copo algorithm for 3M timesteps and below are the training plots.

2. But when I evaluate with the trained checkpoint I am getting success_rate =0 and out_of_road = 100% attached evaluation video. https://user-images.githubusercontent.com/26215090/180390861-f85ed568-baf5-4a9c-8c8f-b2f8a6137aa8.mp4

I have used the bidirlane branch code will all the parameters same, same respawn = true. can you please kindly help me with this issue?

[QuanyiLi]

Hi,

I remember that CoPO uses stochastic policy, so please set exploration=True or some options which enable the stochastic Gaussian policy.

[pengzhenghao]

1. I think the result is quite good! You may ask "why success rate is not climbing high to like 80%?" Please remember the environment time steps used to train is 3M, and you have 2 agents. So you actually use 6M steps to train the PPO policy network. Compared to the Roundabout env where we have 40 agents, the total data usage is 40 * 1M = 40M.

2. Could you please show the code of your evaluation script? What @QuanyiLi said might be one reason. Looking at the video I think it should be the misalignment between training environment's config and evaluation environment's.

[dineshresearch]

Hello @pengzhenghao @QuanyiLi For evaluation I am using the same provided scripts vis.py, vis_from_checkpoint.py and get_policy_function.py from bidirlane branch.
eval_scripts.zip

But I am getting below errors.

python vis_new.py Successfully registered the following environments: ['MetaDrive-validation-v0', 'MetaDrive-10env-v0', 'MetaDrive-100envs-v0', 'MetaDrive-1000envs-v0', 'SafeMetaDrive-validation-v0', 'SafeMetaDrive-10env-v0', 'SafeMetaDrive-100envs-v0', 'SafeMetaDrive-1000envs-v0', 'MARLTollgate-v0', 'MARLBottleneck-v0', 'MARLRoundabout-v0', 'MARLIntersection-v0', 'MARLParkingLot-v0', 'MARLMetaDrive-v0']. Traceback (most recent call last): File "vis_new.py", line 51, in action = policy_function(o, d) File "copo/eval/get_policy_function.py", line 164, in call obs_dict = self.process_svo(obs_dict) File "copo/eval/get_policy_function.py", line 184, in process_svo svo_mean, svo_std = meta_svo_lookup_table[self.model_name] KeyError: 'copo_bidir'

can you please kindly help

[pengzhenghao]

Hi @dineshresearch

I prepared a formal script for evaluating your population from RLLib's native checkpoint. Please check out here:

https://github.com/decisionforce/CoPO/blob/main/copo_code/copo/eval.py

[dineshresearch]

Hello @pengzhenghao @QuanyiLi ]

Currently is their any way that the ego vehicle can identify the bottleneck and know the distance from the bottleneck to its current position?

1. Using LIDAR sensor is it possible, or should I have to use RGB camera and Image processing?

if so can you please explain the process in detail. or any other way you can suggest

[QuanyiLi]

Actually, there are various solutions. The easiest one that I can think is:

1. Access current_map.blocks and find the Bidirection Block whose ID is B
2. Retrieve one lane from the road network of this block
3. Use lane.position(longitude_dist, lateral_dist) to get the global position of the point you are interested in

For example, lane.position(lane.length/2, 0) will return the global position of the middle point of the selected lane. If this lane is in the bidirectional block, then this point will be the central point of this block. After that, you can calculate the distance between this block and the current vehicle position.

[dineshresearch]

@QuanyiLi thanks for response. I understand that using the above steps we can get the bottleneck info and distance however my question is that ego vehicle by using its own sensors (LIDAR/RGB camera) sense the bottleneck and distance. Instead of calculating it from the block info.

[pengzhenghao]

I remember we tried to use lidar to detect the boundary (now lidar is only used for detecting vehicles). Maybe we can restore the functionality in this case

[dineshresearch]

Thanks a lot @pengzhenghao for response. Could you please help me with that code.

[QuanyiLi]

Hi @dineshresearch,

You can update env_config["vehicle_config"] with following dict:

{
"side_detector": dict(num_lasers=12, distance=50, gaussian_noise=0.0, dropout_prob=0.0),
"show_side_detector": True
}

Then, it can detect road edges.

Quanyi

[dineshresearch]

@QuanyiLi I have updated the configuration and able to see the 12 side detectors. However i am unable to understand and make sense of the 12 values obtained. Can you suggest a way to find the bottleneck distance from the log attached containing 12 side lidars data.

[0.16135524213314056, 1.0, 1.0, 1.0, 1.0, 1.0, 0.1560758501291275, 0.17495082318782806, 0.28628072142601013, 1.0, 0.3547708988189697, 0.19210374355316162]tail: car12.txt: file truncated [0.1475025862455368, 0.19042350351810455, 0.4317340850830078, 1.0, 0.2654397487640381, 0.18251855671405792, 0.1747024953365326, 0.22554311156272888, 0.511391282081604, 1.0, 0.2239682972431183, 0.1539975255727768]tail: car12.txt: file truncated [0.1627494841814041, 1.0, 1.0, 1.0, 1.0, 1.0, 0.15529093146324158, 0.1712946742773056, 0.2723306119441986, 1.0, 0.3786896765232086, 0.1971578747034073]tail: car12.txt: file truncated [0.14724311232566833, 0.18851126730442047, 0.4172647297382355, 1.0, 0.26934289932250977, 0.18328484892845154, 0.1742464005947113, 0.22308768332004547, 0.4938293695449829, 1.0, 0.22745051980018616, 0.15477244555950165]tail: car12.txt: file truncated [0.16436900198459625, 1.0, 1.0, 1.0, 1.0, 1.0, 0.15457363426685333, 0.1677277684211731, 0.259249746799469, 1.0, 0.40710583329200745, 0.2028106451034546]tail: car12.txt: file truncated

[QuanyiLi]

This sensor is for detecting the sidewalks, and you can use these features to train your model to detect if it is a bottleneck or not, since the bottleneck has special structures. However, for training this model, you still need ground-truth labels, which can be got from the method I mentioned above. Actually, I wonder why you want to detect it by lidar when you can calculate this distance directly.

[pengzhenghao]

Actually, I wonder why you want to detect it by lidar when you can calculate this distance directly.

Maybe he just want to get a uniform data structure that can also accommodate to other platform / dataset