robodhruv / visualnav-transformer

Official code and checkpoint release for mobile robot foundation models: GNM, ViNT, and NoMaD.
http://general-navigation-models.github.io
MIT License
610 stars 78 forks source link

Seeking Guidance for Simulating Models on Carla #16

Closed chungcode0218 closed 9 months ago

chungcode0218 commented 9 months ago

Dear authors and respected experts,

I am currently looking to simulate models on Carla and I'm seeking guidance. Here's my current environment setup:

I believe I have successfully set up the Carla-Ros-Bridge and Kobuki. However, I'm unsure about how to integrate them effectively with Carla. Do I need to manually create cameras and robots (e.g., Turtlebot2) in Carla by myself, and create nodes to receive image data from the virtual cameras in the CARLA environment?

As I am relatively new to this field, I apologize in advance if my questions come across as rudimentary or offensive. Your assistance would be greatly appreciated.

Thank you.

Dashora7 commented 9 months ago

We used the following repo for CARLA experiments: https://github.com/OATML/oatomobile . This one is fully ROS-agnostic and build a gym-like API on top of CARLA. This system directly spawns sensors, pedestrians, cars, etc via API calls to the simulator's python API wrapper. If you use the oatomobile wrapper, you can mostly work abstracted above sensor management, but you can also add/remove things if needed.

942411526 commented 7 months ago

We used the following repo for CARLA experiments: https://github.com/OATML/oatomobile . This one is fully ROS-agnostic and build a gym-like API on top of CARLA. This system directly spawns sensors, pedestrians, cars, etc via API calls to the simulator's python API wrapper. If you use the oatomobile wrapper, you can mostly work abstracted above sensor management, but you can also add/remove things if needed.

How we control the car in [oatomobile] our predictions with linear and angular that different to Carla

Dashora7 commented 7 months ago

Getting vehicle controls involves using CARLA's API for determining throttle and brake commands. This API runs through a PID controller which accepts a target speed and a desired waypoint. Use from agents.navigation.controller import VehiclePIDController to get the controller. One option is to simply convert your velocities into this format and plug it into the controller to generate an action usable for the agent.

942411526 commented 7 months ago

linear and angular

we have linear and angular without waypoints

942411526 commented 7 months ago

linear and angular

we have linear and angular without waypoints

Getting vehicle controls involves using CARLA's API for determining throttle and brake commands. This API runs through a PID controller which accepts a target speed and a desired waypoint. Use from agents.navigation.controller import VehiclePIDController to get the controller. One option is to simply convert your velocities into this format and plug it into the controller to generate an action usable for the agent.

convert to throttle and brake commands show a bad performer than others

Dashora7 commented 7 months ago

You can generate waypoints by assuming a time delta.

942411526 commented 7 months ago

You can generate waypoints by assuming a time delta.

how to do this ? can you get me some ideas or guidance?

Dashora7 commented 7 months ago

Well using some small time delta, you can compute the angular displacement of the vehicle by multiplying angular velocity times time. In that direction you can then assume linear movement magnitude equal to time times the linear velocity. This should give a rough estimate of where it is. It's not perfect, you'd need derivatives, but a small enough time delta should fix that up.