Merge Videos with different aspectRatios

[omarojo]

I have a list of videos AVAssets. I want to merge them into 1 video with their corresponding audios.

But the videos sometimes are portrait, sometimes square, sometimes landscape (they may have infinite different width and heights). I want the videos to merge and stay aspectFit to the size frame of the first video.

Is this possible with Cabbage ? Im having a hard time understanding your ¨timeline¨ concept.

[vitoziv]

It's possible and easy. Below is the demo code

let item1 = TrackItem(resource: resource)
// Set all item's base content mode to .aspectFit
item1.configuration.videoConfiguration.baseContentMode = .aspectFit
....
item2.configuration.videoConfiguration.baseContentMode = .aspectFit
....
itemN.configuration.videoConfiguration.baseContentMode = .aspectFit

let timeline = Timeline()
timeline.videoChannel = [item1, item2, ..., itemN]
timeline.audioChannel = [item1, item2, ..., itemN]

let compositionGenerator = CompositionGenerator(timeline: timeline)
// Set canvas to first video's size
compositionGenerator.renderSize = item1.resource.size
let playerItem = compositionGenerator.buildPlayerItem()

[omarojo]

ohh interesting.. I will try it that way. :) 👍

[omarojo]

Ok so I tried your suggestion, but it seems like all videos start playing AT THE SAME TIME and in a inverted order

. instead of playing one after the other. I know because I can hear the audio of the first video been overlapped with the audio of the second video. I also get to see a portion of the first video.

And as additional note: Sometimes the exported video is invalid and cannot be played, I feel it might be timing related. It happens specially when merging 3 videos, I need to be able to merge infinite number of videos :(

here is my code.

                let tLine = Timeline()
                var vChannel = [TrackItem]()
                var aChannel = [TrackItem]()
                for clip: G8Clip in self._clips {
                    if let asset = clip.asset {
                        let resource = AVAssetTrackResource(asset: asset)
                        // Create a TrackItem instance, TrackItem can configure video&audio configuration
                        let trackItem = TrackItem(resource: resource)
                        // Set the video scale mode on canvas
                        trackItem.configuration.videoConfiguration.baseContentMode = .aspectFit
                        vChannel.append(trackItem)
                        aChannel.append(trackItem)
                    }
                }
                tLine.videoChannel = vChannel
                tLine.audioChannel = aChannel

                // Use CompositionGenerator to create AVAssetExportSession/AVAssetImageGenerator/AVPlayerItem
                let compositionGenerator = CompositionGenerator(timeline: tLine)
                // Set the video canvas's size
                var firstVideoDimensions = CGSize(width: 1920, height: 1080)
                if(self._clips.count > 0){
                    let firstVTrack = self._clips.first?.asset?.tracks(withMediaType: AVMediaType.video).first
                    firstVideoDimensions = CGSize(width: firstVTrack!.naturalSize.width, height: firstVTrack!.naturalSize.height)
                }
                compositionGenerator.renderSize = firstVideoDimensions

                //Create the Export Session
                if let exportURL = outputURL {
                    self.removeFile(fileUrl: exportURL)
                    if let exportSession = compositionGenerator.buildExportSession(presetName: preset){
                        exportSession.shouldOptimizeForNetworkUse = true
                        exportSession.outputURL = exportURL
                        exportSession.outputFileType = self.fileType
                        exportSession.exportAsynchronously {
                            DispatchQueue.main.async {
                                completionHandler(exportURL, exportSession.error)
                            }
                        }
                        return
                    }

                }

Im attaching 2 Videos (in zip file) that showcase the issue. 1- In this one the landscape video is supposed to be the first one, but it shows the portrait video on top and playing at the same time IMG_7814.mp4.zip 2- In this one, there are 3 videos. the sequence should be: BirthdayCakeVideo, PortraitVideo, AnimatedPhotoVideo(this one has no audio). But instead it plays all of them at the same time, and layered in an inverted sequence. IMG_7813.mp4.zip

It also fails when I try to merge many videos (ie. 6 videos) even 3 fails.

[vitoziv]

tLine.videoChannel = vChannel
tLine.audioChannel = aChannel

// Call this after you set videoChannel. This method will correct the video's start time, base on your order
try! Timeline.reloadVideoStartTime(providers: tLine.videoChannel)

[omarojo]

woah. that worked !! 🥳🤣👍👍 thank you, with just a few lines of code it does what I needed.

Quick question: I noticed that it takes a while to compile/export the final video with the exportSession I guess this is normal. But I wonder how TikTok App does it when mergina and exporting all the clips and showing the final composition almost instantly. I'm thinking I can show to the User the final video using the compositionGenerator.buildPlayerItem() and use that to show a video player with the composition instantly. and then later export to a file letting it take as long as it needs. what u think ?

[vitoziv]

Yes you are right. Now you are a master of video editor. 😜