Regarding /collecting data from stationary cameras

[cpatel245]

Question

What's your question?

Hello, I want to prepare the data-set in the indoor setting. The scene would contain two-three people. Requirements:

• Multiple stationary cameras (most probably two cameras, at opposite corners of the room )
• Three agents/person in the scene, doing basic movements (e.g., walking, standing, extending arm, squatting etc. )
• cameras doesn't move, person needs to walk around a room
• is it also possible to get the skeleton data of the avatar/person?
• is it possible to record data in a such a (described on previous bullets) settings using Airsim?

Include context on what you are trying to achieve

• I want to detect the person (and also the skeletons if it would be possible to get GT from airsim) in the indoor scene, and that's why I want to generate synthetic data-set.

Context details

• I would prefer python to generate/record this dataset.
• I am using Win10, AirSim 1.6.0, Unreal version 4.27.1

[rajat2004]

External cameras might fit your usecase of having stationary cameras, added in #3320. Have a look at the external_camera.py script also. Note that the Recording feature (using R button, or recording API) currently doesn't support external cameras

By skeleton data, you mean the "ground truth" of the pose of the person? There isn't any such capability in AirSim currently, but if there are ways to get that through UE4, then it might be possible to modify the camera implementations to support that, not a trivial task though

[cpatel245]

Thanks @rajat2004 ,

I am currently using Computer Vision mode, and I also need to use depth sensors to generate point clouds from the captured images and depth sensors. But it seems, CV mode does not support adding sensors, since it has to be mounted on a vehicle.

Would it be possible to add sensors in the environment without any vehicles (let's say with the camera itself, so that we can somehow mimic RGB-D camera configs)?

[rajat2004]

Hmm, addition of sensors to the CV vehicle would probably be a good feature. By depth sensors, you mean Lidar right, since you can get depth images from external cameras. One possibility would be using multirotor with ExternalPhysics as the physics engine, then you can have the vehicle fixed at some point without any issue.

As a longer-term improvements which can be done in AirSim, I think #4110 is relevant here, it'll likely require (or good to have) support of sensors on CV vehicle as well. I'll also think on this

Edit: Actually, I'll need to check that sensors don't work on CV vehicle, you're verified that it doesn't work right?

[cpatel245]

I have not used vehicles in CV model since this documentation says so. But as you mentioned, it might be possible to add Multi rotor and fix them at some point.

I wanted to add distance sensor not lidar.

Additionally I tried recording depth and RGB (Scene) images, but simulation crashes while recording Scene Images, Depth images are recorded without any issues. (Already checking existing issues related to this) This error occurs in RenderRequest.cpp

[rajat2004]

That seems misleading, you can specify and create CV vehicle, maybe it was added after the documentation was written. Try specifying a vehicle similar to Multirotor or Car, with the vehicle type as ComputerVision