I've been working on putting together some visualizations for the image agent and I've come up with the following:
This is basically just stitching together the left/center/right RGB images with the BEV route waypoint image that's generated from the Plotter class in leaderboard/team_code/planner.py. I was wondering if you had an idea of how to plot the image waypoints from the perspective of the BEV image? From what I can tell, the BEV image uses GPS coordinates but I couldn't figure out a combination of Converter methods and points_* objects that got it in the right perspective.
The following code snippet (update: link is out of date) is the closest I can get. I think points_world is in GPS coords, and the Plotter code seems to do something similar with the 5.5x multiplier. However, the direction of the image waypoints in BEV frame shown below looks wrong - is there something simple that I'm overlooking?
Update: I got it to work and I think I have a better idea of how world/map/cam are related, though I'm not entirely clear on some aspects. I took points_cam and transformed it to points_map using the appropriate converter method. Looks like 5.5x is the multiplier for going from world (gps?) to map units. I centered points_map at the origin by subtracting (128, 256) which I guess is the ego vehicle's location in the map frame. Then I rotated it to align with the BEV by saving the rotation matrix computed in the ImageAgent.tick method and applying its inverse/transpose. I guess this is the rotation from the world axis to the ego body frame axis or something? Then I multiply the rotated points_map by -1 (reflect over x/y axis) to get it to work - admittedly not clear as to why... and then I centered the points for plotting by adding 256/2 to the coordinates.
This produces the following plot (switched some colors up to make things consistent)
Hey Brady,
I've been working on putting together some visualizations for the image agent and I've come up with the following:
This is basically just stitching together the left/center/right RGB images with the BEV route waypoint image that's generated from the
Plotterclass inleaderboard/team_code/planner.py. I was wondering if you had an idea of how to plot the image waypoints from the perspective of the BEV image? From what I can tell, the BEV image uses GPS coordinates but I couldn't figure out a combination ofConvertermethods andpoints_*objects that got it in the right perspective.The following code snippet (update: link is out of date) is the closest I can get. I think
points_worldis in GPS coords, and thePlottercode seems to do something similar with the 5.5x multiplier. However, the direction of the image waypoints in BEV frame shown below looks wrong - is there something simple that I'm overlooking?Update: I got it to work and I think I have a better idea of how world/map/cam are related, though I'm not entirely clear on some aspects. I took
points_camand transformed it topoints_mapusing the appropriate converter method. Looks like 5.5x is the multiplier for going from world (gps?) to map units. I centeredpoints_mapat the origin by subtracting (128, 256) which I guess is the ego vehicle's location in the map frame. Then I rotated it to align with the BEV by saving the rotation matrix computed in theImageAgent.tickmethod and applying its inverse/transpose. I guess this is the rotation from the world axis to the ego body frame axis or something? Then I multiply the rotatedpoints_mapby -1 (reflect over x/y axis) to get it to work - admittedly not clear as to why... and then I centered the points for plotting by adding 256/2 to the coordinates.This produces the following plot (switched some colors up to make things consistent)