Week2 - AVA: The Art and Science of Image Discovery at Netflix

[kymr]

Title

• AVA: The Art and Science of Image Discovery at Netflix

Summary

• Ava is a set of tools and algorithms designed to surface high quality imagery from the videos.
• Frame Annotations
  • Using archer, they can process a video in parallel.
  • Every frame of video is processed through a series of computer vision algorighms
  • Visual Metadata
  • objective, measurable at the pixel-level.
  • brightness, color, contrast, motion blur
  • Contextual Metadata
  • Face detection
  • Motion estimation
  • Camera shot identification
  • Object detection
  • Composition Metadata
  • a set of heuristic characteristics based on photography, cinematography, and visual aesthetic design.
  • rule of third, depth of field, symmetry
• Image Ranking
  • surface "the best" image candidate
  • Actors
  • Actors play a very important role in artwork
  • One way we identify the key character is by utilizing a combination of face clustering and actor recognition
  • trained a deep-learning model to trace facial similarities
  • Frame diversity
  • there are many different ways to perceive and define diversity
  • the algorithms ability to capture the heuristic variance
  • such as camera shot types, visual similarity (rule of thirds, brightness, contrast), color (colors that are most prominent), and saliency maps (to identify negative space and complexity)
  • Filters of maturity
  • excluded frames containing harmful or offensive elements.

Reference

• https://medium.com/netflix-techblog/ava-the-art-and-science-of-image-discovery-at-netflix-a442f163af6

[kymr]

Words

• imagery plays an incredibly important role in how viewers find new shows and movies to watch
• We take pride in surfacing the unique elements of a story
• As our Original content slate continues to expand
• our technical experts are tasked with finding new ways
• alleviate our creatives from the tedious and ever-increasing demands
• One of the ways in which we do this
• used to broaden the reach of a title
• require an in-depth expertise
• We know through A/B testing
• When it comes to title key art
• it simultaneously presents a very challenging expectation to scale this experience across every title in our growing global catalog.
• AVA is a collection of tools
• designed to surface high quality imagery
• Asking creative editors to efficiently sift through
• is tedious and ineffective
• We set out to build a tool
• Subsequently,
• that meets aesthetic
• As part of our automation pipeline
• to best derive what the frame contains
• in a piece of content
• latent representation of frame
• can be roughly grouped into 3 main categories
• are objective, measurable
• Contextual metadata is comprised of a combination of elements
• to estimate posture and sentiment of the subjects
• to identify camera movement
• compelling still imagery
• the intentions of the cinematographer
• a special set of heuristic characteristics
• Some examples of composition are rule-of-third, depth-of-field and symmetry.
• are ready to begin work for a piece of content
• they are automatically provided with a high quality image set to choose from
• we outline some of the key elements
• all qualifying candidate frames
• we also take into
• highly subjective discipline
• as there are many different ways to perceive and define diversity in imagery
• In doing so,
• which visual elements are most representative of the title
• that we’ve incorporated into AVA
• our creative teams to design stunning artwork
• every single day
• Stay tuned for a follow up blog