search

vishalkrishnads / routewatch

my submission for comma web challenge
https://vishalkrishnads.github.io/routewatch/
0 stars 0 forks source link

Display why openpilot disengaged #6

Open vishalkrishnads opened 3 months ago

vishalkrishnads commented 3 months ago

Currently, routewatch tells the user where openpilot drove and where they drove. It doesn't provide an info as to why openpilot disengaged. This can be a valuable insight into the behavior of a self-driving system as disengagements occur at a point when the AI is no longer seen fit to handle the situation. This can be used to view the edge cases.

The comma API docs state the following

JPEGs are extracted every 5s from road camera video and are exposed at a signed URL in the cloud. To access the JPEGs, first obtain a segment url key from the /v1/devices/:dongle_id/segments response. Chop off the last path component, which represents the segment number. Append secN.jpg where N is the 0-indexed number of seconds since start of the route.

So clearly, it is possible to get such an image from the API. It would be cool if the user could hover over a disengagement point on the map and see why openpilot disengaged. The map has the time in seconds from the start along each GPS coordindate, and so, it should be straightforward to get the image.

vishalkrishnads commented 3 months ago

As of #9 , this feature is working. But it only shows the closest frames available from the disengagement time. As the API only exposes frames every 5 secs. But since this is a single image with a couple of frames, there might be a way to crop them and get individual frames.

Anyway, that's probably a task for another time as it'll overcomplicate things. I'll keep this issue open anyway.