search

microsoft / AirSim

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
https://microsoft.github.io/AirSim/
Other
16.46k stars 4.59k forks source link

Problems for applying AirSim to Visual Inertial Odometry #309

Closed ShukuiZhang closed 7 years ago

ShukuiZhang commented 7 years ago

I know the following questions are a bit too much but that is all that I concerned. I will appreciate it if anyone can answer even one of them. The questions are:

  1. I need scene and depth (or disparity) image with synchronization, but it seems to be an issue for now. See issue. Even that is intractable for now, how can I get the "exact simulator time" when the images and the pose (e.g. position, orientation) are obtained?

  2. How can I get the ground truth intrinsic parameters for the LeftPIP camera. See issue. The point_cloud code works only for default depth window size, but unfortunately, I need larger image than that.

  3. If I understand correctly, the return value of getPosition(), getOrientation() are position and orientations for the "drone", but not for the camera. If I need to know the position and orientation of the camera, I need the relative pose between drone and camera. But seems no clue.

  4. I noticed there is an IMU module, which is really great for me. However, the api does not provide a function to get the imu measurements (e.g. acceleration, angular velocity), is there any sample code that shows how to set it up and work.

My greatest appreciation!!

wingver commented 7 years ago

Hi Zhang, I'm learning VIO and put effort into the same thing you mentioned above. Maybe we can work together to make things done.Can you please contact me, email or wechat？

ShukuiZhang commented 7 years ago

Hi Ye, Nice to hear you are in this area. The problems I posted the other day are progressing! You can keep an eye on new issues because I created new threads for some of them and they get updated! Are you a student?

On Tue, Jul 4, 2017 at 8:54 PM, Ye Yu notifications@github.com wrote:

Hi Zhang, I'm learning VIO and put effort into the same thing you mentioned above. Maybe we can work together to make things done.Can you please contact me, email or wechat？

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Microsoft/AirSim/issues/309#issuecomment-312990426, or mute the thread https://github.com/notifications/unsubscribe-auth/AYBp9AUb63pbvPIE89k1DrzLH7_flIBzks5sKwjkgaJpZM4OJyzB .

-- Best Regards,

Shukui Zhang Ph.D. Candidate Electrical and Computer Engineering, University of California, Riverside

sytelus commented 7 years ago

I've added new APIs that answers 3 out of 4 questions :). The API simGetImages() allows you to get left, right and depth images simultaneously. The API return value is a struct that contains camera position, orientation and time stamp.

For intrinsic parameters of camera, see this discussion. Basically you need to convert depth to camera plan and then use FOV = 90 to create matrix. This code has example of doing many of this things.

sytelus commented 6 years ago

FYI – I just added ability to get camera parameters via APIs in AirSim (1st bullet point below). Here’s more info in our docs.