dmarx / video-killed-the-radio-star

Notebook and tools for end-to-end automation of music video production with generative AI
https://colab.research.google.com/github/dmarx/video-killed-the-radio-star/blob/main/Video_Killed_The_Radio_Star_Defusion.ipynb#scrollTo=oPbeyWtesAoh
MIT License
198 stars 35 forks source link

auto-audioreactivity for variations mode #153

Open dmarx opened 1 year ago

dmarx commented 1 year ago
  1. generate a bunch of variations as usual, potentially at a range of noise levels
  2. either tag each image with the noise level it was sampled at, or (fancier more expensive version) compute inter-image similarities to form a transition matrix
  3. tie the driving signal to the noise level the image was sampled at, or to transition similarity (fun constrained optimization here)