FFmpeg can't combine images & audio to a video, hangs with no callback

[ArsenicBismuth]

This is very similar to the issue I mentioned here: #714 (which I found a workaround by using jpg instead of png). But the behavior are similar: there's no feedback at all (no error, no log, no progress), and the command after it aren't executed (so it's stuck).

As usual, I tested with simpler commands in isolation and they work perfectly: combining images into video, and combining audio with video. Combining them both are not working in FFmpeg.WASM, despite the command I run working perfectly in native FFmpeg.

The command I run is below:

await this.ffmpeg.exec([
    '-framerate', '1',
    '-i', 'images/%03d.png',
    '-i', 'audios/001.mp3',
    '-map', '0',
    '-map', '1:a',
    '-r', '30',
    'output.mp4'
  ]);

The current workaround I have is to do it in two steps, generating the video first then adding the audio. This should mean there's some gap between the WASM implementation and the native one. While that's to be expected, I kinda curious why does the FFmpeg decided to just get "stuck" without any logs or error.

[studio4evr]

this is working for me, where audioElement is the reference to your audio file. you have to make sure you write everything to the ffmpeg memory first.

    console.log('Writing frames to FFmpeg');

//write all the frames into memory here
    for (let i = 0; i < frames.length; i++) {
        const frameData = await fetchFile(frames[i]);
        await ffmpeg.writeFile(`frame${i}.jpg`, frameData);
    }

//then write your audio into memory
    const audioName = 'audio.mp3';
    await ffmpeg.writeFile(audioName, await fetchFile(audioElement.src));

    console.log('Start transcoding');
    console.time('exec');
    await ffmpeg.exec([
        '-framerate', '25',
        '-i', 'frame%d.jpg',
        '-i', audioName,
        '-map', '0:v',
        '-map', '1:a',
        '-c:v', 'libx264',
        '-preset', 'fast',  
        '-crf', '23',       
        '-c:a', 'aac',
        '-b:a', '192k',     
        '-strict', 'experimental',
        'output.mp4'
    ]);
    console.timeEnd('exec');
    console.log('Transcoding completed');