search

deepsense-ai / carla-birdeye-view

Bird-eye's view for CARLA simulator
https://micmarty.github.io
MIT License
182 stars 25 forks source link

how to transform from pixel coordinate to world coordinate? #8

Open kargarisaac opened 4 years ago

kargarisaac commented 4 years ago

Hi,

I have some points in the pixel coordinate frame (for example locations and orientations of cars in the scene) and want to transform it into the world frame. I see that you have location_to_pixel() and apply_agent_following_transformation_to_masks() functions to do it in the other direction (world to pixel). Can you help me to do the inverse?

Thanks

micmarty commented 3 years ago

@kargarisaac Hi, sorry for such a huge delay in reponse :(

I rotated the car on purpose to show you that agent's yaw (relative to the map) is important to properly compute the final carla.Location: image

Point O is the agent's location on birdview (which is always in the middle of an image), but at the same time we know his carla.Location which is agent_vehicle_loc:

https://github.com/deepsense-ai/carla-birdeye-view/blob/f666b0f5c11e9a3eb29ea2f9465d6b5526ab1ae0/carla_birdeye_view/__init__.py#L238-L240

Output array produced by BirdViewProducer has:

O = (h/2, w/2) (in pixels)

Let's say you want to know carla.Location for pixel at X = (20, 10). We know that b = h/2 - 20 and a = w/2 - 10, then c = sqrt(a**2 + b**2) (in pixels). So X is radius_m = c / pixels_per_meter meters away from point O (in CARLA world). At this point we know that our X lies somewhere on a circle with radius=radius_m.

The only thing that now we need is to use yaw information (agent_transform.rotation.yaw) to find the final point on a circle. So you'll need to use trigonometry again to compute that.

Angle between lines b and c is alpha = asin(a / c). 

gamma = math.radians(agent_transform.rotation.yaw)  + alpha # (as far as I remember that yaw was in degrees)

# distances between agent's car to point X (in carla world coordinates - meters)
x_distance = radius_m * sin(gamma)
y_distance = radius_m * cos(gamma)

# final coordinates of X (in carla coordinates)
X = carla.Location(x=agent_vehicle_loc.x + x_distance, y=agent_vehicle_loc.y + y_distance)

Please double check if there'd no problems with angle signs (when X is on the left/right to the car from birdview perspective).

I hope this will do the job. Once again, sorry for not responding so long. I'm planning to focus a bit more on maintaining my stuff 😛. If you're still doing something fun with this package, feel free to submit a PR with some new feature.

micmarty commented 3 years ago

I'll commit this utility method at some point