benetech / VideoDeduplication

GNU General Public License v3.0
34 stars 12 forks source link

Display embeddings with interactive map widget #459

Open stepan-anokhin opened 2 years ago

stepan-anokhin commented 2 years ago

Context

It is hard to display embeddings of a large collection of videos using a single static image. A much better way is to generate a tiles for an interactive map to display embeddings at different zoom level. JS libraries for building maps such as https://leafletjs.com/ also support a lot of useful interactions (like navigating, handling mouse events, drawing polygones, etc.) out of the box, which makes this approach very fertile for adding new features.

Task

  1. Implement a job to generate tiles for an interactive map widget.
  2. Display embeddings using the interactive map widget on the dashboard page if the corresponding embeddings exist.
  3. Show video preview in a popup when user clicks on a video
  4. Add a control to the video-details page to show video's vicinity on the embeddings map.

basic

Further Improvements

Here are some possible developments on top of this task (but out of the scope of this task):

  1. Select an area (e.g. by drawing a polygon) and then application will show you a list of videos from that area polygon

  2. Hover a magnifying glass tool over the embeddings and the application will dynamically show thumbnails and a word-cloud (after semantic text-search will be implemented) of the nearby videos.

  3. Select an area and add all videos from that area to a new collection of videos (after #455 is implemented).