ArdenButterfield / stammer

Recreate any audio track by rearranging the frames of another video
MIT License
434 stars 29 forks source link

Stuck on "analyzing audio" #72

Open 0x5066 opened 1 year ago

0x5066 commented 1 year ago

When running python3 stammer.py SteamedHams.mp4 OnJulySecond1988.wav onjuly.mp4, the video is processed to have it's frames dumped into memory, but when it comes time to read the audio (of what?) and analyzing it, the process just gets stuck doing nothing using up one core and is unkillable by doing Ctrl+C.

Calculating video length
Separating video frames
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kb
frame= 4047 fps=245 q=-0.0 Lsize=N/A time=00:02:48.75 bitrate=N/A speed=10.2x    
reading audio
analyzing audio

This is the current console output. https://files.catbox.moe/79n9ks.wav And this is the file "OnJulySecond1988.wav", am I doing something wrong?

hello10293 commented 1 year ago

Maybe try using a MP3 file instead of WAV?

Firepal commented 1 year ago

@hello10293 This is unrelated to audio formats. When it's "reading audio", it's reading the audio of both files into memory as WAV, regardless of the audio format contained in the input. This is how Stammer matches frames by audio.


Unfortunately, right now the analyzing step can be extremely slow for long inputs, as it is completely single-threaded and does looped operations to match the frequencies of the entirety of both inputs.

It's probably possible to have threads working on chunks of the modulator input, to compare against the entirety of the carrier input. I'd like to experiment with that

Firepal commented 1 year ago

Actually, kinda stumped. There is supposed to be a progress string being printed... I'll have to look into this later, because a two minute carrier and a 30-sec modulator should not exhibit this.

0x5066 commented 1 year ago

I do have ffmpeg installed, so there shouldn't be any issues seeing as it does extract the frames. I did test other formats, exported to 44khz, seeing if it did have problems with 22khz audio... I came around to the same result.

Firepal commented 1 year ago

The input always gets converted to the same sample rate internally by stammer anyway