Image sequence support

[jminor]

Sometimes the media for a clip is a sequence of images.

[repsac]

We've only been using image sequences. I have been simply storing the root directory in "target_url" and then handle loading the sequence in my own logic.

[jminor]

I've seen someone else use a %d or * in place of frame number, like many VFX apps do. These all work fine for the moment in a closed workflow, but some of the adapters, media linkers and cross-company workflows will require that we formalize this more. We should formalize the syntax of the target_url to make it clear how this works.

[bashesenaxis]

fcp xml 7 urls for image sequences are simply a url to the first frame. Though...that doesn't really work all that well, because if I do premiere > xml > premiere, the result is that I only see that first frame. It seems to have forgotten that it was an image sequence. I guess thatll also make it a bit harder to reliably discern whether were dealing with a single image or an image sequence.

But on the topic of frame wildcard, our preference for otio would be in the form of "%04d".

[apetrynet]

Has this been discussed any further? I'm used to and like variations of the "%04d" format my self.

Some alternative ways of indicating sequences (just throwing them out there):

target_url='/path/to/sequence.1001.ext[1001-1050]'
target_url='/path/to/sequence.%04d.ext[1001-1050]'

The following would probably fit in the best to the current code:

target_url='/path/to/sequence.%04d.ext'
start_time=1001
duration=50

And perhaps add an attribute in the ExternalReference class that indicates an image sequence or not. In my work on the Hiero plugin I looked at the MediaSource and MediaFileInfo documentation which gives you an indicator if source is a video container or an image sequence. I would assume other applications supporting image sequences have similar logic so OTIO media_linkers, adapters and plugins could set the "is_image_sequence" flag to True or False accordingly.

In addition to this there's always the option to write functions/methods that analyze the media_reference and determine whether or not it's dealing with a sequence, but not sure if that should be a part of creating a timeline or not. Might slow things down a bit and demands robust I/O routines for all supported platforms.

[apetrynet]

Thought about this some more and threw together a quick suggestion on one way of doing it. It's based on known regex patterns for frame substitutions. So when asking if a media_reference is a sequence it checks for a known pattern in target_url. It's still up to each user to actually convert the target_url into a valid path based on available_range for instance. Please look at this gist and give it a try.

>>> import opentimelineio as otio
>>> media_ref = otio.schema.ExternalReference()
>>> media_ref.is_sequence
>>> False
>>> media_ref.target_url='/path/to/sequence.%04d.exr'
>>> media_ref.is_sequence
>>> True
>>> media_ref.target_url='/path/to/sequence.1001.exr'
>>> media_ref.is_sequence
>>> False
>>> media_ref.is_sequence = True
>>> True

Usage example:

>>> import re
>>> import opentimelineio as otio
>>> media_ref = otio.schema.ExternalReference()
>>> media_ref.target_url='/path/to/sequence.%04d.exr'
>>> media_ref.available_range = otio.opentime.TimeRange(start_time=otio.opentime.RationalTime(1001, 24), duration=otio.opentime.RationalTime(50, 24))
>>> first = media_ref.available_range.start_time.value
>>> duration = media_ref.available_range.duration.value
>>> for frame in range(first, first + duration):
...     print re.sub(media_ref.frame_pattern, str(frame), media_ref.target_url)
... 
/path/to/sequence.1001.exr
/path/to/sequence.1002.exr
/path/to/sequence.1003.exr
...
/path/to/sequence.1050.exr

[alatdneg]

There's a slight problem with using the available range to map to frame numbers, (which I've recently discovered to my cost as I came up with the same solution).

Basically we've got Arriraw files and these have embedded timecodes, (.exr can have the same thing), which is a problem as the available range should hold the timecode, which means I've got no where to store the frames.

I think we definitely need to come up with a proper solution for this as currently we're all rolling our own.

[apetrynet]

Hmmm.. this is true..

It could be up to the media linker to parse metadata from source and apply the correct frame ranges, but that will only be valid in cases where otio does the linking I guess.

I haven't though this one through yet, but another approach may be to introduce some sort of mapping between "source range" and "record range" like EDL's do. Ex: source frame: 0157991 -> record frame: 1001 The way it is now we don't really know if a range is relative to the beginning of the media or a frame number inside the media. If we some how keep both we could keep the timecode metadata from images/video files as "source range" and use whatever frame number passed from applications as "record range"

[jminor]

Here's a python module that might come in handy for summarizing sequences of filenames: https://github.com/rsgalloway/pyseq

[jminor]

Also, we're looking into OpenClip, which may be a more full featured approach to handling media: http://robmoggach.github.io/python-openclip/

[jminor]

Here's another python module that might help: https://github.com/sqlboy/fileseq

[meshula]

FWIW, I wrote a tool called ExifOTIO . https://github.com/meshula/exifotio

ExifOTIO scans a directory for image files with EXIF data. Each image found in that directory is sorted by time. The time of the first image is recorded as the epoch for the track, as a julian date in the track's metadata. Each image is treated as a media source whose duration is the shutter time. The images are concatenated as clips on the track, sequentially, and saved as an OTIO file.

Mostly I'm mentioning it here to record another use case for placing a bunch of images on a timeline, in case it provides any ideas.

[jminor]

Thanks @meshula, that sounds also related to #68.

[ssteinbach]

@jminor in the open clip case, would you expect that there would be special "OpenClipMediaReference" as a schema? or that there would be an openclip data blob in the metadata?

[apetrynet]

@jminor @ssteinbach I might be missing something, but personally I'd like to avoid having to deal with XML objects. Wouldn't it be possible to define some sort of simple media container where we can specify if it's a single or a sequence of files etc? Also, we need to support files like R3D that potentially come in "sequences" as well.

[jminor]

For now we're just researching OpenClip and talking to the people at Autodesk to see how it could work together with OTIO. I agree that putting XML inside JSON would be awkward.

[apetrynet]

Hey! I'm playing around with solving this with a new SequentialReference type which is a subclass of MediaReference. It's just a variation of ExternalReference but contains a frame_range field that stores frame numbers in an opentime.TimeRange. This way we keep the original source Timecode in the available_range and whatever frame numbers the files have in the frame_range I'm not that happy with my naming of the class, but other than that what do you think about this approach?

[apetrynet]

@ssteinbach @jminor Hello again! I've put together a proof of concept plugin for this here. It consists of a schemadef for an ImageReference which is based in ExternalReference and imagesequence_linker media linker. I've added an example file for inspiration. You'll need to dig up an EDL and some images to test it. I can see if I'm able to produce a small sample pack if needed.

I used the rv_session adapter as a test case for output to see how it might work in a real life situation. This quickly revealed that we should consider making the ImageReference type a first class citizen if you feel this is a good direction to go so it may be supported in other adapters. In the RV adapter for instance it needs to detect the ImageReference and ask for frames instead of the "normal" source_range. The map_source_range_to_frame_range method compensates for the potential difference between frame numbers and "TimeCode" and should be called in this case. Please let me know if this doesn't make any sense at all. I've tried to illustrate this in the example file.

Let me know what you think.

[mikekoetter]

FWIW, I know Baselight uses the OpenClip Schema as well

[jminor]

Yay! Thanks for the collaboration on this everyone!