jminor commented 6 years ago

It would be great if otioview had a video display that showed the media that the selected clip's media reference target_url points to.

Note: that this is not the same as playing back the whole timeline, which would be a much larger request.

KarthikRIyer commented 4 years ago

@jminor @ssteinbach @reinecke I'd like to attempt this.

I've tried to implement this by adding a display widget that displays the currently visible frame.

It looks something like this:

otioview

Can I open a PR for this?

reinecke commented 4 years ago

This looks really cool! A bit of discussion on this, but the answers for this probably shouldn't hold back submitting a PR:

How is missing media handled in this?
How are clips with TimeWarp effects shown?
How are transitions handled?
What if the top track is a transparent PNG overlaying a graphic in the corner?

I think this is a really cool and useful feature, but I think it may also be important to be thoughtful about what we will and won't support in the preview and how we communicate that to users.

The risk here is that we make something that's pretty good, gets people excited, and then spend a lot of effort effectively building an NLE compositing engine - this is something OpenTimelineIO definitely doesn't want to do.

Perhaps when there are effects on clips we could do something like overlay a tint color on the playback image along with text like "Effect: Sepia". In the "special" case of LinearTimeWarp effect, it may be best to just omit the clip frames altogether and put up text saying it's not supported?

@jminor @ssteinbach What are your thoughts?

KarthikRIyer commented 4 years ago

@reinecke honestly I hadn't thought about this. These are definitely great points that we need to think about. I"ll submit a PR and make changes as and when things get decided.

I think for missing media we can show a black image with a message. And for transitions maybe just do the same as you suggested for effects?

One doubt. What exactly does flatten_stack do and where would we find a use for that? I was thinking of using this to find which clip is visible but couldn't get that to work with my implementation. Wouldn't the transparent PNG thing be an issue while flattening the timeline?

KarthikRIyer commented 4 years ago

I hadn't considered this to be a proper compositing engine. As you can see when the clip dimensions change I just change the displayed frame and do not center it to the composition dimensions, if any.

jminor commented 4 years ago

One way to approach this without leading to a full video editing application is to show the display only when someone clicks on a clip. By tying the display to the inspector, we could be visually clear that you're looking at the media reference for that clip, without connecting it to the playhead position. This is a bit like the source monitor in an NLE, rather than the record monitor.

KarthikRIyer commented 4 years ago

@jminor so would we display just the first frame from the video? Or have a playhead in the display widget?

jminor commented 4 years ago

Yes, I think showing the first frame of the video would be fine. Perhaps labeling it as "Media Reference" to make it clear that you're showing the source media, not the composited clip? Does anyone else have opinions about this?

reinecke commented 4 years ago

I like this. I feel like this is similar to the other inspectors. The idea is to help you understand what's being referenced by the data model, but to not try and generate pixels for you. In an ideal world, the thumbnail preview would let you play the source (if it's a clip).

I think once we feel we have our time time warps more rigorously defined, then we could think about the source preview allowing you to see a selected frames from the source.

ssteinbach commented 4 years ago

There are two PRs in flight that propose solutions to this. One using gstreamer:

549

and

684

Building a clear specification for this is probably a good topic for a future TSC meeting.

apetrynet commented 4 years ago

This could be very useful, but would it make sense to implement this as a sort of plug-in? That way you can still use otioview if you're missing the underlying video engine. It would also have the benefit of letting you "choose" playback engine such as DJV view or RV in the future which have better image sequence support for instance.

ssteinbach commented 4 years ago

After discussing with the TSC, here is a more concrete proposal:

When clicking a clip widget in otioview:
- in the inspector pane, which currently only shows the json view of the selected clip, having a media preview window that shows the media referenced by clip show up would be handy
- if the reference is an external reference and is a protocol that the backing supports, and the file format is one that the backing can decode, then we can show a preview of that media reference
- if the media reference can't be resolved, presenting a banner or message that explains why the reference can't be read
- if there are effects on the clip, adding a banner noting that the clip has effect on it
to play back the clip just put the entire clip into the inspector
might be nice to have an option to switch between entire clip and source_range
if the source doesn't have any picture data, then the widget is still useful but noting that the source doesn't have picture data is helpful (or if the picture data cannot be decoded)

PySide/Qt seem to have some media widgets, since these are already dependencies of OTIOView, would be great to try and use those as the backing library:

The intent of otioview is as a debugging utility for otio files. Being able to glance at media referenced by an OTIO clip, even without compositing or effects, would still help confirm assumptions about the data.

jminor commented 1 year ago

A lot has changed since this issue was filed. Perhaps adopting tlRender, OpenRV, etc. as a recommended OTIO playback tool would address this without complicating otioview?

AcademySoftwareFoundation / OpenTimelineIO

otioview video display #280

549

684