Audio Reactive Audio Generation

[Trentonom0r3]

Describe the solution you'd like Several Options/Techniques available. I've Tested a few;

1. Auto Generation at transition points (beat kicks, etc;)
2. Audio Reactive Video2Video
3. Audio Reactive Txt2Video (defacto deforum)
4. Reactive Overlay generation

Describe alternatives you've considered N/A

Additional context Would be useful for music videos and edits, which I do a lot of.

[AugmentedRealityCat]

There are some options to use sound to drive effect parameters directly in After Effects, but most of them are commercial and expensive, like Beat Edit.

Free and Open-Source solutions would be more than welcome.

If you do make such a plugin, I would consider making it usable at large as an independant After Effects plugin, this way even people who are not using Stable Diffusion would have a use for it.

Another option is to use the oldschool way: scripting expressions directly in After Effects.

[Trentonom0r3]

There are some options to use sound to drive effect parameters directly in After Effects, but most of them are commercial and expensive, like Beat Edit.

Free and Open-Source solutions would be more than welcome.

If you do make such a plugin, I would consider making it usable at large as an independant After Effects plugin, this way even people who are not using Stable Diffusion would have a use for it.

Another option is to use the oldschool way: scripting expressions directly in After Effects.

Indeed! Since there is a custom pseudo effect, you can pick whip the parameters to audio!

I'm also working on a few custom Python scripts for this, which will be useful to the extension as well as to anyone who wants to use it that doesn't have Adobe products