Create new map

[dongyy1148]

Hi, I want to try some new map that not included in LGSVL simulator, so I download some city map from unity asset and want to follow the map annotation tutorial to add map annotation, but I am stuck on the first step. The "window" button doesn't have "Map Tool Panel". However, when I open the San Francisco map , there is "Map Tool Panel". Why does this happen?

[martins-mozeiko]

Are you opening downloaded map in Simulator project? Map Tool panel is custom code that exists in our simulator. If you open map in different project it won't have this item because code does not exist there. You should open scene/model files in simulator project.

[martins-mozeiko]

Create new folder under Assets, then copy files for new scene (.unity, .fbx, .mat and whatever else it has) into this folder. Then double click .unity scene from this folder to load it.

[dongyy1148]

Thanks for the reply! However, there is no response after I press "create temp map waypoint" button in the downloaded map. But it will add waypoint if I press the button in the map your simulator includes. Even though I made new waypoints, I cannot make lane between two points when I press Make line. Could you help solve it? Besides, I also tried to make a new ego vehicle and I follow the tutorial video this time. But I find that in the video there is a folder named "newvehicle" under assets/prefabs folder, but I cannot find it in my simulator in the unity.

[martins-mozeiko]

Simulator creates these waypoints by doing raycast in middle of screen. This raycast must hit some collider that has Layer "Ground And Road". So basically you need to make sure your map has MeshCollider component added to roads. And these road objects are assigned to "Ground And Road" layer.

Prefabs -> New Vehicle is just a new folder we created where to put files (mesh, textures, etc..) for new vehicle. Name and structure of this folder does not matter, you can create your own folder and use it in whatever way you want.

[dongyy1148]

Got it! But based on the video, I need to match the FL FR in WheelColliders with Element 0 in Vehicle Controller. For the model below, the truck has its four wheels not in the format of WheelColliders. I cannot easily drag to match them. How can I fix it? There is also the same problem for the FL_parent. Does it mean that I have to create the new ego vehicle with the exact format like the new vehicle? If so, how can I find the format like vehicleMeshes, WheelColliders and so on? Thanks!

[martins-mozeiko]

Current vehicle dynamics we have assumes only two front wheels and two rear wheels. If you need more wheels you'll need to modify VehicleController.cs to add more wheels. And then change actual dynamics logic to make these wheels to spin. Search the file for axels to update all necessary places.

[dongyy1148]

I mean that where can I find WheelColliders, Lights, DriverCamera (in the figure) in the assets to add to the new vehicles? Because in the video, it needs to drag these assets to match with the property in the right windows.

[dongyy1148]

Hi, Could you help on how can I get these files(mesh, textures, etc..) under the "LGSVL_vehicle"? I cannot find "LGSVL_vehicle" (reference vehicle) in my asset. Thank you so much!

[EricBoiseLGSVL]

@dongyy1148 ,

Hello and let's see if I can help.

As per the video, you can copy elements (Wheel colliders, lights and sensors) from the existing vehicles and modify to fit your new vehicle. E.g. Duplicate wheel colliders in old prefab, drag into new vehicle hierarchy, center root transform to new vehicle, move child wheel colliders to match new wheel meshes or make new ones, adjust wheel collider parameters to fit needed dynamics and finally drag the new references to the controller class public variables in Unity inspector. This is the quickest and easiest way to create custom vehicles that work in the Simulator.

Another way is to create your own. Wheel colliders, Lights and Cameras are Unity components. We add scripts to modify these components to work in the simulator. There is a button in the inspector of any empty object to Add Component. E.g., for the wheel colliders, add an empty game object named WheelColliders as a child of the new vehicle. Then add one empty child object to the WheelColliders. Highlight that object and press Add Component button in the inspector. Search and add Wheel Collider. Then just duplicate once you have the parameters you need. Use the old vehicles to check Layers, Tags and scripts. If you need extra parameters or new functionality, you will need to edit or extend the simulator scripts. Finally drag the new references to the controller class public variables in Unity inspector.

Meshes and textures need to be created outside Unity. We had 3D artists create those vehicle meshes and textures and were imported into the simulator. Look at Maya or Blender for 3D modeling and Photoshop for texture work. If you want the meshes for the existing vehicles, look at the Mesh Renderer component on the game object, it references the mesh in the project. If you want the Texture that exists in the project already, look at the Mesh Renderer, then the material on that renderer. If you select the material, you can then find the texture that the material references.

I hope this helps,

-Eric

[dongyy1148]

Hi Eric,

Thanks a lot for your help! I want to try the first method you mentioned. However, I cannot find the ego vehicle's prefabs. Could you tell me where I can find the existing vehicles? Thanks!

[EricBoiseLGSVL]

No problem, glad to help.

Assets/Prefabs/AgentCandidates/

[dongyy1148]

The vehicle included in AgentCandidates are XE-Rigged-xxx, which is not exactly the same as the one showed in the video. For example, In XE vehicle, there are collider not the WheelColliders in the vehicles, and it also doesn't have lights. Does it matter? Besides, when I drag the files from XE vehicle to my vehicle, there is a warning like

[EricBoiseLGSVL]

Yes, the video was a modified LGSVL ego vehicle to make the video more clear. The components were organized in the picture you posted to help separate the main objects needed and renamed. Unless the code references the game objects by name, you can rename anything you want. (I'm pretty sure all references are by public variables not name). The child parent order is important though. Some scripts look in parent or children for the code needed.

Also yes. By removing or reordering a prefab in Unity, you will break the old prefab instance (!!! in the scene !!!) not the prefab in the project. If you break the instance and then apply that change to the old prefab, you will overwrite the prefab in the project and that can be bad. Be sure to make a new prefab and add components to that and save the new prefab, not the old.

[dongyy1148]

Hi Eric,

Thanks for the help! I have built a vehicle and I have added my vehicle to the AgentPrefabs array as the eleventh vehicle. However, I cannot find it in the LGSVL simulator. Do you have any solution for this?

[EricBoiseLGSVL]

Be sure you add the new prefab from the project panel, not the scene. Also be sure to Add to ROSAgentManager Agent Prefabs in the project panel as well. This should work, if not then please send a screenshot.

[EricBoiseLGSVL]

The prefab will be at the bottom of the list when you pick agent in menu scene

[dongyy1148]

I have done it like this. Do I need to do anything after I drag my truck to element 10? This is the screenshot in LGSVL simulator.

[EricBoiseLGSVL]

That should work. 1) Are you building the binary or running in Editor? If building, are you building in Unity or command line? 2) Are you dragging the new vehicle to the AgentPrefabs list from the project or the scene? 3) Are there any errors when you run in editor?

[dongyy1148]

1. I run the LGSVL simulator directly to see the result.
2. Yes.
3. I am not sure what do you mean that running in editor. What editor do you refer to?

Besides, are FL_PARENT, FR_PARENT... necessary? I haven't added them to my vehicle, and I also can't find them in the XE-Vehicle. And I have no idea about what these parents are. Could you help explain it? Thanks!

[EricBoiseLGSVL]

1) Are you pressing play in the Unity Editor or are you running command line to build the Simulator, then running the executable? If you are building the simulator using Unity, it will fail 3) The unity editor. Open StaticConfig.unityscene and press play in the Unity editor. Menu.scene will load. See if the prefab is in the list.

Those names are from the 3d artist and are used for the 3d mesh of the wheel to rotate correctly. The truck mesh should have a centered holder node to rotate the wheel mesh. It's what the VehicleController.cs Axles Right/Left Visual rotates when the wheel collider rotates.

[EricBoiseLGSVL]

Also, while running in Unity editor, you can see if there are errors in the console panel. Better to debug there instead of a build and have to try to read a error log

[dongyy1148]

Hi Eric, Is this the way that running command line to build the Simulator? And I can see my NewTruck if I run simulator in Unity editor, but still cannot see it through the simulator executable. I think I didn't build it rightly in the command line. This is how I build it in the command line, and there is a error that "Couldn't set project path to: /home/isuzu/opt/Unity/Editor/.", so I remove the line "-projectPath .". And then it started building but didn't stop. Do I build it rightly?

Besides, I got some errors in the console panel. How can I fix them? Thanks a lot!

[EricBoiseLGSVL]

@martins-mozeiko can help with the build errors.

DriverCamera.cs error - you are missing the FixTo.cs script or a public reference in that script. VehicleController.cs error - you are missing the mesh wheel transforms in the axle array DriverCamera.cs driverCameraPosition error - on the Driver camera there are missing transforms for the driver camera positions.

Be sure to have all public references assigned to the new vehicle and it's components on all scripts. When you duplicate, Unity keeps those references until you delete the old prefab from the scene, then they are missing. That is what I am doing at the end of the video, reassigning all the public references to the new prefab.

Once you have all errors fixed, then create a build

[martins-mozeiko]

@dongyy1148 You need to set -projectPath to location where is your Unity project, for example: -projectPath /home/isuzu/simulator

[dongyy1148]

@martins-mozeiko Thanks for the reply! I will try it later. @EricBoiseLGSVL I have fixed two errors, but there are still two errors.

1. I think I have already added all the camera positions.

2. I have no idea about this one.

Thanks again for your patient!

[EricBoiseLGSVL]

1. It's odd you are getting this error still line 99 is... fixTo.fixTo = driverCameraPosition; It seems that fixTo is null but it's reference is found in Start(). You can try enabling the FixTo script and Apply change to the new prefab (it will disable after Init anyway). If that doesn't work you can make FixTo public and assign that in inspector as well. I've tested and it finds it whether it's enabled or disabled and even if you leave the FixTo.FixTo ref null. Are you getting this error after error 2? If so, always start with the first error in console and ignore latter ones. If you null or error in Unity it tends to fail badly after the first and the latter errors logs are not reliable.

2. On the root transform of your new prefab, look at WindshieldMaterialQualitySwitcher.cs. Be sure to clear the InspectRenderers array that no longer references the old prefab.

[dongyy1148]

Yep. The error2 is before the error1. I have fixed error2, but there is a new one...

[EricBoiseLGSVL]

1) On the root transform, be sure: a) VehicleInputController.cs TargetRosEnv is set to the correct bridge type b) AgentSetup.cs TargetRosEnv is set to the correct bridge type c) AgentSetup.cs NeedsBridge array has all sensors that need bridge (see video) d) All sensors have correct bridge type. The sensors copied are set to the bridge type the prefab was setup for, e.g. LGSVL agent was setup for LGSVL bridge type and all sensors as well.

2) Be sure to have the latest simulator code, this error can happen on an older version of simulator code

[dongyy1148]

Thanks for the help! Now I only have one error that I have tried enable the FixTo script and it doesn't work. How to make FixTo public and assign that in inspector?

[EricBoiseLGSVL]

Great! In DriverCamera.cs line: 17 change: private CamFixTo fixTo; => public CamFixTo fixTo; Save, compile and drag to the now exposed ref in the inspector.

[dongyy1148]

Hi Eric, I have revised the file. And then the fix to.sc is missing for the truck model. Other vehicles (XE-apollo) also miss the fix to.sc now. Then I did the build command like ./Unity -batchmode -nographics -silent-crashes -quit -buildDestination ./build/simulator -buildTarget Linux64 -executeMethod BuildScript.Build -projectPath /home/isuzu/Desktop/simulator/Assets -logFile /dev/stdout. But the fix to.sc file is still missing, so I am wondering how can I compile the DriverCamera.cs. Could I just drag the file to the inspector? Thanks!

[EricBoiseLGSVL]

I see the issue. DriverCamera.cs is not supposed to be on the root vehicle prefab. I needs to be on the Driver Camera, which is a child object of the root. The scripts I see from your image are all root prefab scripts. Please pull any of the old vehicle prefabs into the scene and look how the driver camera is setup.

[dongyy1148]

Thanks for the advice! Now, there is no error showed in the Unity editor, and I did the build command like ./Unity -batchmode -nographics -silent-crashes -quit -buildDestination ./build/simulator -buildTarget Linux64 -executeMethod BuildScript.Build -projectPath /home/isuzu/Desktop/simulator/Assets -logFile /dev/stdout again. And use the LGSVL simulator executable to open the LGSVL, however, my truck still doesn't show in the UI.

[martins-mozeiko]

-projectPath should be path to where Unity project is, not Assets folder. It should be -projectPath /home/isuzu/Desktop/simulator.

Try deleting ./build/simulator folder completely before running build. This way you will know when build fails - the output won't have executable to run.

[dongzh1148]

@martins-mozeiko Thanks a lot for the help! I can see my truck in the LGSVL simulator now. @EricBoiseLGSVL Since my truck's size is different from the XE-Apollo vehicle, I drag wheels assets (FL, FR...) to match my truck's wheel meshes (wheel parents) in the Unity editor. And their places are almost the same in the Unity editor. However, after I choose my truck in the Apollo, the wheels' positions change a lot, and the truck starts to oscillate. If I push up key, the wheels will rotate around an axis above the road and the truck don't move. I think I set the position wrongly in the Unity editor, but I don't know how to align them. Could you give some hints? Thanks a lot!

[EricBoiseLGSVL]

Excellent, the new vehicle is in!

Ok, let's get the wheels working correctly. Please screen capture the Unity Hierarchy panel with the truck mesh children wheel meshes expanded. I need to see how the model is setup. Select the main Collider and the wheel collider before capture so I can see the Unity colliders alignment on the mesh.

like so...

[dongyy1148]

The wheel mesh in the truck model is pretty simple. It only haves wheels but no Pivot children like the XE-Apollo car. For the main Collider and wheel collider, I copy from XE-Apollo vehicle and adjust their sizes based on the truck model.

[EricBoiseLGSVL]

I think I see some issues.

1) The BoxCollider needs to be reduced in size in the Y and moved up. For you it should be at the top of the truck bed and just at the bottom of the truck cab lower bumper.

2) The wheel mesh pivots on the old ego meshes are very important. They are moved and rotated at runtime to the wheel collider position or Pose. Make sure the WheelCollider holder object is at 0,0,0 local position relative to the root game object. Also, match the wheel collider radius to the mesh radius

Another cause could be if the pivot is not right or the pivot has odd rotations in Maya, then when the Simulator tries to rotate and position the wheel mesh you get the crazy positions from your capture. It should look like this... (mesh should be z or blue arrow forward, this is rotating during play mode)

This is an example of the NPC truck with physics colliders in Simulator

[dongyy1148]

Thanks for the detailed explanation!

1. Does this look right?

2. If I set them to be 0,0,0. The wheel collider doesn't match with the wheel meshes. Should I adjust the FL, FR..'s transform to match them? Besides, the wheel meshes don't have the radius value, is it okay that I match the radius using eye measurement?

3. I need to add pivots to my truck model right? Do you think it would be easier to modify on a NPC truck? Thanks again!

[EricBoiseLGSVL]

1) perfect 2) yes, move the wheel collider transform to match, then eye measure the radius 3) not sure. The important thing is that the wheel mesh pivot is centered and when the code moves the mesh to match the wheel collider, it aligns correctly. Some models use a pivot parent object to hold the detail meshes in an object but still can be controlled by code or animation at this control node.

[dongyy1148]

I think I followed your tutorial and fixed all the problems. But the truck showed in the LGSVL simulator is still odd. And it is odd that only the front wheels are in the wrong places, the rear wheels are normal. But rear wheels are below the road, the front wheels are above the road.

I want to ask that is this FL_parent enough (without pivot...) Or it should take the same format like XE-Apollo, like:

Besides, I also check the model for NPC truck, and I find that the meshes for wheel is also very simple for NPC truck and it also doesn't have the pivot children under FL_parent.

[EricBoiseLGSVL]

Which transform are you using for the VehicleController.cs Axels leftVisual and rightVisual? When you select FLParent in editor and press 'W', does the unity transform gizmo (the xyz arrows handle) align with the center of the wheel? If so, then use that as the Visual transform for the script not Front wheel Grp.

If it is still off and that is centered then the mesh is wrong and needs adjusted. Then I would just use the LGSVL truck (it looks exactly the same :) )

The pivot object is just used to center complex meshes to a single pivot transform or node. It is used a lot in animation and rigging. The focus is to get a centered transform for the wheel mesh to match the collider. Names on meshes don't really matter. You can even add your own empty game object to the prefab and child the wheel mesh to it.

Another thing I noticed is that you added a lidar to the top of the truck. If you are using if with an AD stack the lidar will hit the truck and not work correctly (it will post collisions on itself). You might need another approach.

[dongyy1148]

Yes. The gizmo is at the center of the wheel. Is this blue arrow forward like you said before?

[dongyy1148]

Besides, I found one problem that should I make the transform of the wheel collider and wheel meshes to be the same? If so, which one should I use as the correct transform? This is the transform of the wheel collider: This is the transform of the wheel meshes: Thanks!

[dongyy1148]

I create a new one based on the NPC truck. And I have succeeded to import it into the LGSVL simulator. Besides, I can control it using the keyboard in the simulator now. So I want to do some simple tasks connecting with Apollo using my new truck. However, the truck doesn't move this time after the Apollo gives the path planning. These are some screen shots from the Cyber_monitor. It seems that path planning and control module work well, but the chassis doesn't have the throttle or other information. Could you help look at it? Thanks!

[martins-mozeiko]

Does lidar_timestamp value changes? If not, then this probably is same issue as #195. This seems to be a bug in Apollo, we are still checking how to fix it.

[dongyy1148]

The lidar_timestamp value changes all the time. It is odd that I can control by keyboard, but Apollo cannot control it.

[martins-mozeiko]

Do you have VehicleInputController component added to vehicle game object?

You can try adding Debug.Log loggin in VehicleInputController.FixedUpdate function to check autoInputAccel and autoSteerAngle variables - these should match values Apollo is sending in control command.

[dongyy1148]

@martins-mozeiko I have added VehicleInputController component to the vehicle. Could you give more details about how to add Debug.Log loggin in Void FixedUpdate function? Thanks a lot! Besides, if I add the Debug.loggin, will the autoInputAccel and autoSteerAngle be outputed in the command window during the simulation or somewhere else?

[EricBoiseLGSVL]

@dongyy1148 Inside the FixedUpdate() at the end of the method, Debug.Log("AutoInputAccel: " + autoInputAccel + " / AutoSteerAngle: " + autoSteerAngle);

This output is in the playerlog, https://docs.unity3d.com/2019.1/Documentation/Manual/LogFiles.html