Aerial View using CARLA ( Drone view)

[rak7045]

Hi @germanros1987 Hello All! I kindly need your inputs and suggestions regarding this issue. This post is lengthy and please read it.

I have set the sensor (Camera) attached to vehicle with carla.Location(x=-3, z=50), carla.Rotation(pitch = -20) as the orientation of camera. The parameters of the sensor i.e., Altitude (Z-axis) should be not less than 50m and FOV = 69°. Later, the FOV might be changed, but the altitude parameter value is fixed.

However as you can see, due to FOV and Altitude the objects in the images are small and unclear. If there are bikes far away in the map those seem like a dot in the image and getting bounding box to it will be difficult. For getting a detailed view trying to import vehicle models using Blender or Maya? Will this be helpful? My opinion is if I import also the objects will be small. What would be your thoughts on this?

I have used Depth Camera information and test semantic LiDAR(not combined but two different scripts) for occluding and filtering the distant objects.

• In test semantic LiDAR, I set the attribute "RANGE = 1500m" which shouldn't be the case. This is a blunder mistake as the LiDAR max range will be ~150m.
• While using Depth Camera I set the "max_dist = 3500m" where the objects are not occluded and not filtered as well. There was also distorted images as shown below.

My questions are as follows:

1. Since, I am in the drone view the objects will appear small. How can I zoom in or get detailed view of the objects in the map?

2. How can I eliminate the false negatives (i.e., there is an object but no bounding box to it)?

Could someone help me with this? or Please provide me some documentation or references?

Thanks in advance Ravi Teja

[brscholz]

Hi Ravi Teja,

what image resolution does your camera use - from the image you postet I guess it's FullHD? With small objects in a larger distance this all may boil down to a sampling problem.

A higher resolution at runtime and downsampling in your postprocessing routine might improve your results regarding semantic segmentation. You would only have to make sure that objects of interest always "win" a cluster of pixels over non-interesting objects.

Do you have reference images from the drone you're trying to depict in the simulation or is your project all virtual?

Best regards Robert

[rak7045]

Hi Robert, Thanks for your reply. Yes, those images are full HD (1920*1080).

At the moment it is virtual. We have taken VisDrone dataset as reference. The VisDrone dataset is like BEV (i.e., the drone having the camera pointing downwards) which is not the exact in our case. We are collecting the synthetic dataset from CARLA at the moment.

For producing the synthetic datasets I had those problems. I would like to know what might be the possible solutions to get detailed view of the objects and also eliminate the false negatives. I have several things running in mind and a lot of confusion.

Thanks and Kind Regards Ravi Teja

[brscholz]

Hi Ravi Teja,

I'm not sure what you mean with possible solutions to get a detailed view of the objects. With a camera of this resolution at this distance to the target, you will very likely not be able to get a detailed image of e.g. a bike frame. It's just a matter of pixels that get hit by the light emitted by your object of interest. If your target drone camera uses this resolution, I'm afraid that little bunch of colored pixels is what it the camera will see of the bike, too.

If you want larger, more detailed images, you would have to go closer. If you want to maintain the size of the view area but gain more details, you would have to increase the camera resolution.

Best regards Robert

[rak7045]

Hi Robert,

Consider the above pictures as example. As we can see the whole map the objects are not at all clear. Suppose later we want to train using this data the network couldn't distinguish the class (car or truck). And also we couldn't see if there is a bike far away. This is the major problem.

If you want larger, more detailed images, you would have to go closer. If you want to maintain the size of the view area but gain more details, you would have to increase the camera resolution.

• Yeah, the drone can fly around 50-100m as per regulations. So, our altitude is set to 50m. If I increase the resolution I might get more details, but then how can I eliminate false negatives as objects are distant from the camera? (This question arises next) :(

I have came across a concept in Unreal Engine called " Cinematic Depth of Field" where we can blur background and foreground and zoom in to the area of interest. This is not acceptable for our case.

So, I would like to know are there any such concepts so that I can zoom in to certain area have clear details ( I mean clear identification of objects in the map).

My point is since the camera can see whole map, the objects appear small and couldn't able to get bounding box to it(These are the main concerns I need to get rid off). Generating synthetic dataset is the crucial part for a neural network to learn . So need some guidance on this.

Hope you understood my query. Sorry if I mess it up.

BR Ravi Teja

[ricardodeazambuja]

For the people interested in using drones, UAVs, quadcopters, etc inside CARLA, I modified the original CARLA-ROS-BRIDGE and created a set of ROS2 packages that allow just that. Everything runs from docker if you just want to test it. I am looking for someone using Unreal because my system is missing an actor created inside Unreal editor to give it a body as currently it has the physics, but no body and, therefore, no collisions without manually detecting it using the sensors. On the other hand, I can just use the standard CARLA docker image (it is currently using the 0.9.13, so texture randomization is available). Here is the link to the repo https://github.com/ricardodeazambuja/ros2_quad_sim_python

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.