search

autowarefoundation / autoware.universe

https://autowarefoundation.github.io/autoware.universe/
Apache License 2.0
952 stars 620 forks source link

Test the new AWSIM Labs parking lot map #8342

Open armaganarsln opened 1 month ago

armaganarsln commented 1 month ago

We have recently created a new parking lot map here:

The link is here: https://github.com/autowarefoundation/AWSIM-Labs/releases/download/v1.2.0/awsim_labs_ParkinArea.zip

We would like the following new Parking Planning PRs to be tested before merging to confirm they work in the AWSIM Labs simulation environment.

Expected outcomes:

cc. @xmfcx @maxime-clem @Aysenayilmaz @mitsudome-r @takayuki5168

maxime-clem commented 1 month ago

@armaganarsln thank you for providing the map. We will prepare a report comparing the parking behavior before/after all the PRs. We will do our best to prepare this report by the next Planning & Control working group. If you do not mind, we will evaluate all PRs together as they were split to make the reviews easier, but the behavior improvements is mostly noticeable with all PRs.

I also understood that during the working group it was agreed that you would provide test scenarios for scenario_simulator_v2. Is this still planned ?

mkquda commented 1 month ago

@armaganarsln

In response to your request, the following PR has been tested using the provided parking lot map: (this PR includes all the changes from other PR's as well)

Few things to note:

lanelet2_map.zip

Test 1: (L parking next to parked vehicle)

https://github.com/user-attachments/assets/b3f7a9b0-337d-4303-b9c6-c4fa34891c19

Test 2: (L parking between 2 parked vehicles)

https://github.com/user-attachments/assets/0d7cf1cd-a47e-4e2a-b4c6-59fb2c861aa5

Test 3: (Angled parking between 2 parked vehicles)

https://github.com/user-attachments/assets/92a010e3-c6cb-4b3f-89f3-05cce437706f

Test 4: (Parallel parking)

https://github.com/user-attachments/assets/22813f29-8af4-447f-bb85-82a91d8ff661

Test 5: (L parking with obstacle)

https://github.com/user-attachments/assets/071bb9e8-dbb8-44cd-9b8d-a0345a95ee8b

armaganarsln commented 1 month ago

@armaganarsln thank you for providing the map. We will prepare a report comparing the parking behavior before/after all the PRs. We will do our best to prepare this report by the next Planning & Control working group. If you do not mind, we will evaluate all PRs together as they were split to make the reviews easier, but the behavior improvements is mostly noticeable with all PRs.

I also understood that during the working group it was agreed that you would provide test scenarios for scenario_simulator_v2. Is this still planned ?

@Aysenayilmaz started working on the scenarios for AWSIM Labs Scenario-based testing but @mkquda already made the above tests and there are changes made to the lanelet. So first thing we should test the current planner with the above lanelet and further discuss the details of why those changes to lanelet are needed and if these changes are necessary. I have made the parking area with a relaxed setting and using the biggest parking lot standard available so I am not convinced that there are changes required to these sizes. I will like to discuss the details during the planning meeting to learn from you (planning team) why these changes must be made and why there is no other way to park.

armaganarsln commented 1 month ago

I have checked the tests above and apparently, they are made in the planning simulator not in AWSIM therefore could you also publish your results in AWSIM Labs for us to see the performance. For example the area that you parked has a vehicle in that place . image

xmfcx commented 1 month ago

You can find the AWSIM Labs binary in Releases section:

xmfcx commented 1 month ago
  • Due to the duplicate lanes drawn on top of each other (which I assume are to allow two way movement), It was difficult to run the simulations smoothley using the provided lanelet map, in many cases the mission planner fails to find a route.

I understand you've removed one of the overlapping lanes to make it work, we can fix it in the future but still this is unrealistic. This parking lot has those spaces bidirectional like most real parking lots. We can leave this part to future.

  • In current Autoware configurations, costmap_generator has buffer size of 1.0 m around object polygons, and freespace_planner has a safety margin of 0.5 m added to ago vehicle shape for collision detection, so free space path planning requires at least 1.5 m between ego and detected objects. In the layout of the provided map the spacing between the parking slots is not sufficient to satisfy these settings when trying to park next to a vehicle.

This is not realistic as well. The parking lot was created by following regulations and standards.

Due to above 2 points I modified the provided lanelet map to remove the duplicate lanelets, and increase the spacing between the parking slots. The modified lanelet map is attached below.

Instead of modifying the parking spaces, please consider changing the default parameters of the Autoware.

If the current safety margins in the Autoware are very unrealistic, try reducing them and let's focus on improving the safety without losing the ability to park in this lot.

image

Parking when there is this much space in all those parking spaces should be possible. They are all easy parking spots with lots of room for error.

Aysenayilmaz commented 1 month ago

Park Testing Report

I've conducted a series of tests on parking algorithms using AwsimLabs and Autoware. I'm on the main branch in Autoware ( a742e824366ac63f18fc2d304ad9e648132098a4 ) and this pr (https://github.com/autowarefoundation/autoware.universe/pull/8089) has been merged into the current main branch. Below are the observations and results:

Test 1: Parallel Parking Attempt

https://www.youtube.com/watch?v=f6jN62Wny-I

https://github.com/user-attachments/assets/f5172cd8-386a-4d53-b89a-2fce9c1866a6

Test 2: Goal Pose Behind the Vehicle

https://www.youtube.com/watch?v=tmlBhRD0oFc

https://github.com/user-attachments/assets/2a38cd06-c523-4b0f-89c7-40d6a21b8f6b

Test 3: Parallel & Angled Parking

https://www.youtube.com/watch?v=JdTxU79LLG4

https://github.com/user-attachments/assets/ff04d7d2-db45-40ab-a258-407193deb58f

Test 4: L Parking Issue

https://www.youtube.com/watch?v=0bYq64MQT3w

https://github.com/user-attachments/assets/be7fde2d-685d-4b52-9f6a-58151e3edf5c

cc : @xmfcx @armaganarsln @mkquda

mkquda commented 1 month ago

@armaganarsln @xmfcx @Aysenayilmaz

I conducted the following tests in AWSIM with modified costmap_generator & freespace_planner parameters to reduce safety margins.

Test 1: (L parking next to vehicle)

https://github.com/user-attachments/assets/45980444-3013-4ffd-892e-774081190894

Test 2: (L parking between 2 vehicles)

https://github.com/user-attachments/assets/09610da8-de1f-49db-9ce7-c1b30213b9cb

Test 3: (Angled parking next to vehicle)

https://github.com/user-attachments/assets/c0b26150-917e-432e-8a10-586d5743ab15

Test 4: (Parallel parking)

https://github.com/user-attachments/assets/ec9bb7dd-32a9-4912-b8d9-ea98dc4a1a5a

xmfcx commented 1 month ago

@mkquda thanks for the tests. Could you also post the parking behavior for these tests before your changes?

So we can see how it has improved with your additions.

Aysenayilmaz commented 3 weeks ago

I created the tests conducted by @mkquda using the scenario simulator, and ran them with AwsimLabs and ss2 together. I'm sharing the results as a youtube playlist. I'm also adding the yaml files of the scenarios to the drive.

_Youtube playlist : https://www.youtube.com/playlist?list=PL-g7xKnyjxjggO6xgFJ_n3Labd8OFk93J_

_Scenarios files : https://drive.google.com/drive/folders/1CbLekZoej3XICKTSgkTt0zQyZ_g8_lhA?usp=drive_link_

Aysenayilmaz commented 3 weeks ago

Park Testing Report

I've conducted a series of tests on parking algorithms using AwsimLabs and Autoware. I'm on the main branch in Autoware ( a742e82 ) and this pr (#8089) has been merged into the current main branch. Below are the observations and results:

Test 1: Parallel Parking Attempt

* I attempted a parallel parking maneuver.

* The generated path was not suitable for parallel parking.

* Resulting in a failed parking attempt.

https://www.youtube.com/watch?v=f6jN62Wny-I 2024-08-14.15-09-44.mp4

Test 2: Goal Pose Behind the Vehicle

* I set a goal pose in a parking spot behind the vehicle.

* While the vehicle stayed within the parking area, it failed to complete the parking maneuver successfully.

* This might require further tuning, but I am not sure about the specifics.

https://www.youtube.com/watch?v=tmlBhRD0oFc 2024-08-14.15-55-26.mp4

Test 3: Parallel & Angled Parking

* When setting a goal pose for parallel parking, the system successfully generated a path.

* For angled parking spots, the system failed to generate a path.

* The error _"Component_state_diagnostic "_ was encountered.

* The vehicle could perform autonomous driving for parallel parking but could not set a goal pose for L-shaped or angled parking.

https://www.youtube.com/watch?v=JdTxU79LLG4 2024-08-14.16-21-28.mp4

Test 4: L Parking Issue

* In another location, I attempted to set a goal pose for L-shaped parking.

* The system did not exit the stop mode when the goal pose was set, and no path was generated.

https://www.youtube.com/watch?v=0bYq64MQT3w 2024-08-14.16-26-28.mp4

cc : @xmfcx @armaganarsln @mkquda

I had to change 'obstacle_threshold: 101' and 'expand_polygon_size: 0.0' to encounter the issue I mentioned, but I know this means it won't detect obstacles. I did it just to draw the path and test.