OpenTimelineIO / OpenTimelineIO-AVFoundation

Conversion and Compatibility between OpenTimelineIO objects and their CoreMedia and AVFoundation counterparts
Apache License 2.0
23 stars 2 forks source link
avfoundation coremedia macos opentimelineio post-production swift video-editing

OpenTimelineIO-AVFoundation

OTIO Reader Icon image
Realtime playback of OpenTimelineIO files with the sample app OpenTimelineIO Reader:

Note

This library is under heavy development!

General

This Swift Package extends OpenTimelineIO's Swift Bindings to provide functionality for Apple platforms.

The goal is to enable easy interoperability OpenTimelineIO and AVFoundation objects in a correct, lossless and useful manner to enable real world workflows.

This library should be compatible with the following Apple platforms:

but to date has only been extensively tested on macOS

Quickstart for Users

Quicktart for Developers:

OpenTimelineIO to AVFoundation:

Assuming you have a basic AVPlayer setup, this will let you import a .otio file with basic jump cut editing. See roadmap for transitions / effects.

    do {
        if
            let timeline = try Timeline.fromJSON(url: url) as? Timeline,
            let (composition, videoComposition, audioMix) = try await timeline.toAVCompositionRenderables()
        {
            let playerItem = AVPlayerItem(asset: composition)
            playerItem.videoComposition = videoComposition
            playerItem.audioMix = audioMix

            self.player.replaceCurrentItem(with: playerItem)
        }
    }
    catch
    {
        print(error)
    }

AVFoundation to OpenTimelineIO:

Assuming you have succssfuly created an AVCompostion - this will export a basic .otio file without effects or transition metadata. See roadmap for transitions / effects.

    do {
        let timeline = try compositon.toOTIOTimeline(named: toURL.lastPathComponent)
        try timeline.toJSON(url: toURL)
    }
    catch
    {
        print(error)
    }

OpenTimelineIO Extensions

Core Media Extensions

AVFoundation Extensions

Compatibility:

OTIO Format Compatibility Matrix

OTIO Formats Status Notes
OTIO :white_check_mark:
OTIOD :white_check_mark:
OTIOZ :x: Planned

NLE Compatibility

NLE Import from NLE Export to NLE
Davinci Resolve 18.6 :white_check_mark: :white_check_mark:
Davinci Resolve 19 :white_check_mark: :white_check_mark:

Project Validation

Sample Project Import to AVFoundation Export from AVFoundation
ALab Trailer :white_check_mark: :white_check_mark:
AWS Picchu Edit :white_check_mark: (Decompress .otioz zip file and open the bundled .otio file) :white_check_mark:
OTIO-OC-Examples :white_check_mark: :white_check_mark:

Video Format Compatibility

Generally if Quicktime or Final Cut Pro X Can support it, it should just work.

Video Formats Status Notes
H.264 :white_check_mark: Native
HEVC :white_check_mark: Native
Apple ProRes :white_check_mark: Requires Apple Pro Video Formats for some variants
Apple Intermediate Codec :white_check_mark: Requires Apple Pro Video Formats for some variants
MXF wrapped Pro Res :white_check_mark: For developers: Requires MTRegisterProfessionalVideoWorkflowFormatReaders() and VTRegisterProfessionalVideoWorkflowVideoDecoders() to be enabled
MXF Wrapped DNx :white_check_mark: Requires Apple Pro Video Formats for Avid DNxHD® / Avid DNxHR®. For developers: Requires MTRegisterProfessionalVideoWorkflowFormatReaders() and VTRegisterProfessionalVideoWorkflowVideoDecoders() to be enabled - only some DNx variants work:
MXF Wrapped DNxHD_36 :white_check_mark:
MXF Wrapped DNxHD_80 :warning: (codec specifically gets subsampled down to 1440 x 1080)
MXF Wrapped DNxHD_115 :white_check_mark:
MXF Wrapped DNxHD_175 :white_check_mark:
MXF Wrapped DNxHR_SQ :white_check_mark:
MXF Wrapped DNxHR_LB :white_check_mark:
MXF Wrapped DNxHR_HQ :white_check_mark:
Image Frames :x: Requires custom compositor
Image Sequences :x: Requires custom compositor
Raw Formats (BRaw, Red, etc) :x: Requires you to have SDK - manage decode and roll your own custom compositor

Dependencies

Roadmap

FAQ

RationalTime uses double as numerator and denimonator for rational time calculations. CMTime uses Int64 and Int32 for its value and time base. In order to get most accurate conversions, OpenTimleineIO-AVFoundation uses looks at the number of decimal places in a RationalTime representation and scales it to a whole number to maximally represent the same ratio without rounding or truncating which would happen with naive Double <-> Int casting.

See our Unit tests for conversion notes. In general, for integer frame rates (ie 120, 60, 30, 25, 24) there is zero loss conversions. For non integer frame rates like 59.97, 29.97, 24.98 (23.976) there may be very minor differences in the conversions as of today (our Unit testing uses an accuracy factor of 0.00000000001 )

If you have ideas on lossless conversion, PR's are welcome! This was a first pass :)

Credits:

OpenTimelineIO and TimecodeKit, and Ozu.ai for supporting this effort.