Record in sync with Music playing like Tiktok

[omarojo]

You see how in Tiktok or Instagram Stories, you can select background music. Then everytime you press the REC button, the music plays so that the user is able to lip-sync along with the music.

Well with nextlevel the recording clips part is done, Once I merge all clips into 1 single file, I replace the whole audio track with the music. The problem is that, the music is not in sync with the video. I think it is because I use an AVAudioPlayer to play/pause the music when pressing the REC button, and I believe AVAudioPlayer may have micro delays when playing and pausing (it is just for the user to listen to the music and lipsync, because at the end we are replacing the audio track entirely anyways)

So my guess is.. using an AVAudioPlayer just to play music and expect it to be in sync with the recorded video is just the right way to go.

Do you have any insights or suggestions ?

[alexjameslittle]

You should use AVMutableVideoComposition for creating editable videos like this I believe, you will get a lot of control of when the music will start and end, volume and more. You can also preview your compositions in real time without waiting for merging multiple clips. This doesn't seem to be directly related to next level so may be best to do some more research on AVFoundation if you need greater control. I recommend taking a look at apples archive code sample linked here:

https://developer.apple.com/library/archive/samplecode/AVCustomEdit/Introduction/Intro.html#//apple_ref/doc/uid/DTS40013411

[jeremyschoenherr]

@omarojo did you come up with a solution to this using clips from NextLevel or did you have to do something else? i'm doing something similar and fear i may be in the same spot. after i merge the clips via NextLevel, i create an AVMutableVideoComposition with the video from that merged video and then audio from a different source, but its a bit out of sync. any insight would be appreciated :)

[omarojo]

I ended up using https://github.com/VideoFlint/Cabbage

To merge all clips + music on top.

Creating all the VideoCompositions myself was too much hassle and complex. The Library above makes it fairly easy.

But yes.. lipsyncing is super hard... because having ex. 20 independent video files that you created with NextLevel for example..  when merged.. they kinda have this tiny delays in between each clip...

Lipsycing is just something I havent been able to nail down. On Jan 14, 2020, 7:46 PM -0600, Jeremy Schoenherr notifications@github.com, wrote:

@omarojo did you come up with a solution to this using clips from NextLevel or did you have to do something else? i'm doing something similar and fear i may be in the same spot. after i merge the clips via NextLevel, i create an AVMutableVideoComposition with the video from that merged video and then audio from a different source, but its a bit out of sync. any insight would be appreciated :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[AdnanAliShabir]

@omarojo I want to record video with sound like TikTok but the problem is my microphone voice is also recorded so how i can fix it to only play the selected sound? Thank you!