How to set Music for entire Timeline or individual Video tracks

[omarojo]

Hi im back :)

So previously I successfully implemented your suggestions to merge videos with their corresponding audio tracks. Now Im wondering if it's possible 2 things.

1- Is it possible to define separate audio for each video with specific range of that audio ? example:

let tLine = Timeline()
var vChannel = [TrackItem]()
var aChannel = [TrackItem]()
//VIDEO Tracks
... trackVideoItem1, trackVideoItem2, trackVideoItem3...
//AUDIO Tracks
        let musicUrl = Bundle.main.url(forResource: "HumansWater", withExtension: "MP3")!
        let musicAsset = AVAsset(url: musicUrl)
        let resourceA = AVAssetTrackResource(asset: musicAsset)
        let trackAudioItem1 = TrackItem(resource: resourceA)
... same for trackAudioItem2, trackAudioItem3...
But how do I specify the start-end and duration of those tracks.  ??

tLine.videoChannel = [trackVideoItem1,trackVideoItem2,trackVideoItem3]
tLine.audioChannel = [trackAudioItem1, trackAudioItem2, trackAudioItem3]
try! Timeline.reloadVideoStartTime(providers: tLine.videoChannel)

Currently the above creates an unreadable video.

2- The other question is, If it's possible to define 1 music track for the entire video composition. example:

let tLine = Timeline()
var vChannel = [TrackItem]()
var aChannel = [TrackItem]()
//VIDEO Tracks
... trackVideoItem1, trackVideoItem2, trackVideoItem3...
//AUDIO Track for everything
        let musicUrl = Bundle.main.url(forResource: "HumansWater", withExtension: "MP3")!
        let musicAsset = AVAsset(url: musicUrl)
        let resourceA = AVAssetTrackResource(asset: musicAsset)
        let trackMusicItem = TrackItem(resource: resourceA)
But how do I specify the start-end of the audio (trimming the audio)

tLine.videoChannel = [trackVideoItem1,trackVideoItem2,trackVideoItem3]
tLine.audioChannel = [trackMusicItem]
try! Timeline.reloadVideoStartTime(providers: tLine.videoChannel)

Is it possible to set 1 audio for everything? and what happens if the audioTrack is shorter than the entire video composition or the video composition is shorter than the audioTrack, would it repeat the audioTrack ?

Thanks in advance :) :)

[vitoziv]

You may try Timeline.audios. Set your audio to it and specific the audio's timerange

[omarojo]

Hi @vitoziv so for the last week I tried using Timeline.audios. After lot's of experimenting I figured out how the API works. But I had some issues. For example, Im trying to set a music asset as the background audio for the entire video composition, and IF the video composition is longer than the music asset, then I repeat the audio from the beginning by calculating how many times the Duration of the music asset can fit the Video.

video: [------------------------------] audio: [----][----][----][----][----][--]

In the example above the audio asset can fit 5 times in the video and a 6th time with a reduced timeRange. That's where I got problems with the calculations. And the end I had to substract 0.05 to the last timeRange END value, otherwise it would fail.

                let tLine = Timeline()
                var vChannel = [TrackItem]()
                var aChannel = [TrackItem]()
                for clip: G8Clip in self._clips {
                    if let asset = clip.asset {
                        let resource = AVAssetTrackResource(asset: asset)

                        // Create a TrackItem instance, TrackItem can configure video&audio configuration
                        let trackItem = TrackItem(resource: resource)
                        // Set the video scale mode on canvas
                        trackItem.configuration.videoConfiguration.baseContentMode = .aspectFit
                        trackItem.configuration.audioConfiguration.volume = 0.0
                        vChannel.append(trackItem)
                        aChannel.append(trackItem)
                    }
                }
                let musicUrl = Bundle.main.url(forResource: "HumansWater_Short5", withExtension: "m4a")!
                let musicAsset = AVAsset(url: musicUrl)
                self.musicCabbageResource = AVAssetTrackResource(asset: musicAsset)

                self.musicCabbageResource!.prepare(completion: { (status, error) in
                    //Note: When Audio is longer than the whole video composition is when it can fail. That's why we Loop the audio
                    if status == VFCabbage.Resource.ResourceStatus.avaliable {
                        print(">> Total VIDEO DURATION: \(self.totalDuration.seconds)")
                        print(">> Music File Duration: \(musicAsset.duration.seconds)")

                        let numOfLoops = self.totalDuration.seconds/musicAsset.duration.seconds
                        let numOfLoopsRoundedUp = numOfLoops.rounded(.up)
                        var sumPartsTotals = CMTime.zero

                        var endS = CMTime.zero
                        for i in 0..<Int(numOfLoopsRoundedUp) {

                            let mResource = AVAssetTrackResource(asset: musicAsset)
                            //Audio Trim
                            let start = CMTimeMake(value: Int64(0.0 * 600), timescale: 600)

                            if(i == Int(numOfLoopsRoundedUp)-1){ //is the last chunk of audio
                                let lastChunkTimeFrac = numOfLoops.truncatingRemainder(dividingBy: 1) // ex 1.5 will give 0.5
                                let lastChunkTimeSecs = musicAsset.duration.seconds * lastChunkTimeFrac //music from 0 to this value

                                endS = CMTimeMake(value: Int64((lastChunkTimeSecs-0.05) * 600), timescale: 600)

                            }else{
                                endS = CMTimeMake(value: Int64(musicAsset.duration.seconds * 600), timescale: 600)
                            }

                            mResource.selectedTimeRange = CMTimeRange(start:start , end: endS)

                            let part_mTrackItem = TrackItem(resource: mResource)
                            part_mTrackItem.startTime = CMTimeMultiply(musicAsset.duration, multiplier: Int32(i))

                            print("start:\(part_mTrackItem.startTime.seconds) - totalPart:\(mResource.scaledDuration.seconds)")
                            sumPartsTotals = CMTimeAdd(sumPartsTotals, mResource.scaledDuration)
                            tLine.audios.append(part_mTrackItem)
                        }
                        print(">> Total Duration of Audio Parts:\(sumPartsTotals.seconds) - should be same as Total Video Duration")

                        tLine.videoChannel = vChannel
                        tLine.audioChannel = aChannel

                        try! Timeline.reloadVideoStartTime(providers: tLine.videoChannel)
                        try! Timeline.reloadAudioStartTime(providers: tLine.audioChannel)

If I remove the - 0.05 the calculations are correct. But the composition FAILS, because I think that for some reason the tLine.audios total duration end up larger than the video.

For now it works if I do - 0.05. but do you have a better suggestion ?

Thanks anyways. As always great Library. !! 👏👏👍

[vitoziv]

Nothing wrong with your code. Maybe it's because of the double calculation, try do less double calculation.