Monocular Depth Estimation

[danielrhuynh]

1. Investigate SOTA dnn-based monocular depth estimation models and test performance on our GPUs
2. Integrate model into ROS 2 node
3. (separate task) Generating point clouds from depth estimation outputs

Old ros2 node for reference: https://github.com/WATonomous/deepracer_ws/pull/2

June 13 2024 - Moving Forward:

• Fix sourcing issue
• Take in rosbag of nuscenes data and get real time predictions
• Integrate into WATO infrastructure

[danielrhuynh]

June 27 (Skipped a week for midterms):

• No progress, reassigning to mark if he is able to pick this up
• Reassigned to Mark

[danielrhuynh]

July 4

• Mark synced with Justin + Hank + Jessica
• Main blocker: machines are down

[danielrhuynh]

July 11 2024 - Blocked by SLURM :/

• We have repos to get distance estimations (CPP integration)
• Python solution has metric depth for pointcloud generation
• Deciding on cpp + python implementation (for speed)
• we will focus on python rn so we can get it implemented for association and then optimize later on

[danielrhuynh]

July 18 2024

• Depth anything v2 has dependancy issues
• Can't connect to SLURM???, tried to run locally hasn't worked yet
• Actionables after dependancies:
• Node already Implemented just need to build out DockerFile to integrate into our infra

[danielrhuynh]

July 25 2024

• Depth estimate wrapper node is made just needs testing
• image -> pointcloud takes a long time to render
• start looking into a cpp solution now that we know how de is implemented
• Look into monocular performance at night
• Look into de integration for association