question about sim_control mode

[ideasplus]

Hi, I want to run perception, prediction, planning and control module by replaying a rosbag and use dreamview to see the car's behavior. But Apollo seems not support control-in-loop simulation. So I find the sim_control mode which can move the car accornding to the planning result. But when I open sim_control mode and necessary module and then replay a rosbag, it doesn't work for me, dreamview will jump between two scenarios.

My question is what can we do in sim_control mode? Can we replay a rosbag in sim_control mode to test perception, prediction and planning module simultaneously?

[daohu527]

Ref to how-can-i-test-planning-algorithms-offline

You can test planning algorithms in sim control, I think your problem may be that you play a repeated topic, please confirm the frequency of the topic. and just open planning and routing module.

[ideasplus]

Thanks for your repley! By looking at relevant code, I think sim_control is a tool for testing planning algorithms and it publishes fake location and chaiss messages as the input of planning module. But I saw the following description in a paper about sim_control:

We perform the case study using the simulation feature provided by Baidu Apollo, called Sim-control, which is designed to allow users to observe the AV system behavior at the driving decision level by replaying collected real-world sensor data traces

Is the description here correct? If so, what should I do to run end-to-end test under sim_control mode?

[daohu527]

In my opinion

As you said, they simulated the "driving decision level".

The simulator has certain limitations by replaying the sensor data actually collected. When the behavior of your own car changes, real-world data also needs to change.

For example, in the real world, your car has to move forword, but when you change the planning algorithm, your car may change lanes. The data in the real world can't adapt to this change but still play the old data, so you can't use this method in a long time simulator.

advice

1. If you want the end-to-end simulator use deep reinforcement learning, you can try carla or lgsvl
2. If you want to simulator the apollo modules, you can use a cloud simulator provide by apollo team. Apollo Simulation

[ideasplus]

@daohu527 Thanks for your helpful advice!

I want to clarify the mearning of end-to-end test I mentioned before, It means use real sensor data such as point cloud as input and test the apollo pipeline under sim-control.

In their paper, they claim that use modified real 3D point cloud as input to oberve the car's behavior through dreamview under sim-control. Because we want to reprocude their experiment result, But I really don't know how to do according to my current understading. So I look for some help.

[daohu527]

You can think of "sim control" mock a real physic car , which following exactly the path planned by the "planning" module.
If you don’t care about the car dynamics model, you can follow this steps below to simulate.

steps

1. choice the map and car in dreamview
2. open the "perception" "prediction" "planning" "routing" "transform" and "sim control" modules.
3. play the bag and be careful to check not play the TOPIC "chassis" "localization" use the cmd

   cyber_record play -f xx.record -k chassis localization    //xx.record is your record bag

4. observation and verification

[ideasplus]

@daohu527 Hi, I have tried according to your method, but the car will not move on the dreamview. I don't know what happend. By the way, I want to say that sim_control will public fake localization and chassis messages, but whether the messages match the played record file? The perception module needs /tf topic, but whether the topic in the record should be reserved in this situation due to the topic is published by localization module.

[daohu527]

I want to say that sim_control will public fake localization and chassis messages, but whether the messages match the played record file?

The car just following the path your planning module give. And the planning module give the path by the record environment.

The perception module needs /tf topic, but whether the topic in the record should be reserved in this situation due to the topic is published by localization module.

The /tf is published by record file and is "static".

[Moon-XUChengyuan]

@daohu527 Thanks for your helpful advice!

I want to clarify the mearning of end-to-end test I mentioned before, It means use real sensor data such as point cloud as input and test the apollo pipeline under sim-control.

In their paper, they claim that use modified real 3D point cloud as input to oberve the car's behavior through dreamview under sim-control. Because we want to reprocude their experiment result, But I really don't know how to do according to my current understading. So I look for some help.

I am facing the exact same problem as yours, that using real sensor data as input and test the apollo pipeline. Is there any progress? Thanks a lot !!

[Capri2014]

@daohu527 Hi, I have tried according to your method, but the car will not move on the dreamview. I don't know what happend. By the way, I want to say that sim_control will public fake localization and chassis messages, but whether the messages match the played record file? The perception module needs /tf topic, but whether the topic in the record should be reserved in this situation due to the topic is published by localization module.

Apollo has simulation environment for you to test your algorithms open accessible to public at: bce.apollo.auto. The simulation environment supports the "control-in-loop" vehicles of different brands (MKZ and Coolhigh) with different dynamic models you can use (Automotive rule-based or Learning based). You can run your test from prediction all the way to control. (We only provide "perfect perception" in this environment as of now.) Also if you are interested in more tech details behind it, you can refer to this paper we published for more information: https://ieeexplore.ieee.org/document/8968102

[sjiang2018]

Hope our answer resolved your question. We will close the issue for now. If you have any additional question, please feel free to open a new issue. Our engineer team are more than happy to help that.

Thank you for supporting Apollo!