search

Netflix / photon

Photon is a Java implementation of the Interoperable Master Format (IMF) standard. IMF is a SMPTE standard whose core constraints are defined in the specification st2067-2:2013
Apache License 2.0
234 stars 77 forks source link

Audio duration is not an integer length for video frame rates 29.97 or 59.94 #209

Closed go4shoe closed 5 years ago

go4shoe commented 6 years ago

If I use 30,000/1001 or 60,000/1001 video with 48 kHz audio I can only make the audio stream the exact duration of the video stream if the video frame count is divisible by 5.

It seems a fairly harsh rule considering that video may consists of multiple segments.

For example - if my video consists of multiple segments then all video segments have to be aligned on a 5 frame boundary.

Example for one segment:

Video 29.97 with 5,378 frames would require at audio 48 kHz with 8,613,404.8 samples. I obviously can not encode 0.8 audio samples.

Currently my choices: add two video frames to get to 5,380 or remove three video frames to get to 5,375.

What do you guys think about having the audio integer sample count within +/- one sample of the fractional calculated audio sample count?

palemieux commented 6 years ago

@go4shoe Why not use markers instead of segments? In other words, where did the requirements for segments come from? Segments are intended to be essentially independent parts of the timeline.

What do you guys think about having the audio integer sample count within +/- one sample of the fractional calculated audio sample count?

This was ruled out from Application 2/2E as yielding unpredictable results.

go4shoe commented 6 years ago

Thanks for your response and pointing me towards the correct specification (Application 2/2E). I am not sure how markers are going to help me. If my video consists of two individual files 29.97 fps and both files have a frame count that is unaligned to 5.

palemieux commented 6 years ago

I am not sure how markers are going to help me. If me video consists of two individual files 29.97 fps and both files have a frame count that is unaligned to 5.

Individual resources can have a duration that is not a multiple of 5. It is the sequence whose duration must be a multiple of 5. For instance, the following is permitted:

Segment
- Audio Sequence
  - Track File Resource (48 kHz)
    - duration = 299
  - Track File Resource (48 kHz)
    - duration = 7709
- Image Sequence
  - Track File Resource (29.97 fps)
     - duration = 2
  - Track File Resource (29.97 fps)
     - duration = 3

[Edited to address https://github.com/Netflix/photon/issues/209#issuecomment-337382902]

go4shoe commented 6 years ago

Thanks again for your inside - I suspect a typo in your above comment:

Do mean the "Segment" must be a multiple of 5?

Your above comment says the "Sequence" must be 5 aligned.

cconcolato commented 5 years ago

Closing this issue as it seems that your problem is with the spec. Photon does verify, per the spec, that the durations of sequences in a segment match, see https://github.com/Netflix/photon/blob/master/src/main/java/com/netflix/imflibrary/st2067_2/IMFCoreConstraintsChecker.java#L310