Import HDmap into simulator

[daohu527]

I want to import hd map into lg simulator, maybe the openstreet format first. And then I will try apollo hd map format.

1. It's useful for end to end test in self-driving. If you first create a map in the simulator and then export it as a hd map format. Actually the maps we use in apollo or autoware are labeled with other systems and they cannot be unified. So we need to import the label map into simulator without any 3d geometry structure. We will automatically generate it. Then you can adjust the house and street map in the simulator.

2. By the way, when did you release the version?

   Note that we'll have a big change in our project structure to support new Unity's HDRP rendering pipeline. But map annotation format won't change much, we are keeping it more or less the same.

101

[daohu527]

Create 3D model

I found some ways to create a new map from the OSM data format:

1. CityEngine. You can import data from 2D map (OSM). Then set some rules to generate buildings, roads, plants and so on. When you finished the 3D map, you can export the models and import them into Unity.

   By the way, The most effect way to create a new unity scenes is just replace the environment by the models export from cityengine?

2. Utymap. Which is a open source project to generate city 3d models by OSM data. utymap

   You can also look at OSM unity

   Create HD map

   When you import the city 3d models into unity, you can use the HDMap Tool to make and export the HDMap to apollo or autoware.

Questions

If you want to import hdmap directly, you must create a function that automatically generates roads and buildings. I think it might be possible to import OSM data first. And make an adpter to change hdmap to osm map. Then the map of the game is the same as the real map. You don't need to create a HD map twice.

[martins-mozeiko]

I think there are different uses cases for map import that can require very different implementations.

One use case is when user wants to import HD map for testing their path planning or routing. They do not necessary care about buildings or great visuals as long as basic road network is created so that NPC vehicles can use it. I would say this is a good way to start import - just create hdmap annotations and put everything on a plane (so car can drive). For a lot of use case this can be enough.

Next step - to add buildings actual road surface, with or without textures could be done later. Getting actual data is tricky part. One option is to generate them procedurally. This could give you very nice visuals, but requires more work. Another would be to use real world data. But that sometimes is not enough if it has low precision or low quality. Running perception can require high quality/precision data that something like OSM might not have.

[daohu527]

Good advice! The simulation is used to test autonomous driving, and we need to be as real as possible, not completely real. In fact, we need more scenarios. A completely real world does require a lot of simulation work. We need accurate location information, buildings, plants, textures, etc.

One use case is when user wants to import HD map for testing their path planning or routing. They do not necessary care about buildings or great visuals as long as basic road network is created so that NPC vehicles can use it. I would say this is a good way to start import - just create hdmap annotations and put everything on a plane (so car can drive). For a lot of use case this can be enough.

This is really what we want, apollo provides "sim_control", which is a simple simulation test planning module. But it can't detect collisions, and NPC is not smart. It is much easier to achieve this in Unity.
First, import HD maps into Unity and only generate roads without buildings. Then send the traffic light status and NPC location information to Apollo or Autoware. Finally, control the car to drive and detect collisions, whether or not exceed the road. By the way, is there an API that provides NPC location and speed information?

Next step - to add buildings actual road surface, with or without textures could be done later. Getting actual data is tricky part. One option is to generate them procedurally. This could give you very nice visuals, but requires more work. Another would be to use real world data. But that sometimes is not enough if it has low precision or low quality. Running perception can require high quality/precision data that something like OSM might not have.

Currently, I think the better way is to use OSM data to generate buildings, although the accuracy may not be enough. Otherwise, it is difficult to have other good data. Where do we get real data?

[martins-mozeiko]

By the way, is there an API that provides NPC location and speed information?

Yes, Python API can get and set any npc location and speed.

There is no easy way to get real data with good quality. We are internally currently working on pipeline to generate map from collected point cloud & camera. But that won't have good accuracy for buildings. But it will have good quality road - accurate geometry and color information.

[daohu527]

Good, I will try to do it.
Currently, does the simulator support changing the angle of view? If just test the planning module, I think the overhead view is a good choice.

[martins-mozeiko]

Not during runtime. But if you run inside Editor you can use Scene view to switch camera to orthographic projection and keep camera position above the car pointing downwards.

[daohu527]

Additional, How does the NPC know the lane and the direction? They just move by program or they have AI and know the lane, the direction and the traffic light?

Basically, we have map annotations for each scene, and this map annotations have these information: lane, direction, traffic light, traffic sign, and etc. NPCs know the lane and the direction using the map annotations and drive around following the traffic rules we have. By AI, if you are asking if NPCs are detecting lanes or traffic lights using its own sensors, they are not for performance reasons. Only ego vehicles have sensors like a camera or LiDAR and can be controlled by AD stack.

[daohu527]

@martins-mozeiko I have created a demo, but there are some questions to discuss. This is a map rendering.

1. I created the map by lane, which means I didn't use street_one_way4lane material for rendering. Not all adjacent lanes are equal in length. It is difficult to deal with this situation, including making the adjacent lane starting points consistent and equal in length. I don't know if HD maps can guarantee these. The safer method is based on the lane is going to be rendered, so it seems that the road is a bit strange, but it will not affect the effect.
2. More work is needed with junctions, crosswalks and others. Maybe create a layer that covers over road is an easy way.

[martins-mozeiko]

Additional, How does the NPC know the lane and the direction? They just move by program or they have AI and know the lane, the direction and the traffic light?

NPC's does not have advanced AI - they just follow annotated lanes, stop at traffic lights (which are annotated, so they know exact position & state of light). Then they randomly decide to turn or drive straight. NPCs do not use any sensors like EGO vehicle does.

Did you take street_one_way4lane material from SanFrancisco scene? This material is very hardcoded to roads in SF scene. If you want to create new different roads, then you will need to use texture manipulation and 3d editor software (Photoshop and Maya or similar) to adjust mesh and texture so it looks good. Unity cannot do this for you.

[daohu527]

I render based on the lane instead of the road. And it seems to work, I will upload a branch to see the effect

[daohu527]

@martins-mozeiko I have already submit a pull request. Could you pls make a review #243