search

WICG / video-editing

Other
65 stars 8 forks source link

Why does it not make sense just to create playlist? #13

Open ytakio opened 4 years ago

ytakio commented 4 years ago

I'm sorry that I don't understand context for this proposal. But it makes sense for me to provide framework like a playlist for editing and playing. Playlist means that it just a list to have target blob and time range to be played.

It seems to be heavy for client side encoding. If this proposal provide some playlist and API to edit and play, server side encoding may be available by using edited playlist as needed.

Of course, playing may have some pause at bound of discontinuity point. If user want to make it smooth, then they just capture via MediaRecorder by using playlist.

nigelmegitt commented 4 years ago

The idea of a playlist is very powerful, since it is a thing that can incorporate multiple media types, and could itself be serialised and deserialised for reuse.

palemieux commented 4 years ago

Expanding on https://github.com/WICG/video-editing/issues/13#issuecomment-637585981, it looks like the current API is sufficient in the simplistic case where the playlist consists of a sequence of clips of the same kind:

image

I do not think however it can express the (increasingly common) case where the playlist consists of multiple kinds of resources (audio, video, timed text...) that are not coterminous:

image

In the example above, the timeline cannot be expressed as a sequence of blobs.

To account for such case, the API could be extended to allow a given time interval to contain multiple blobs to be reproduced in parallel:

image

This approach however loses semantics associated with the playlist. For example, unless the link between blob and media is preserved, splicing artifacts may be present at blob boundaries even if the source media was continuous.

It would therefore be good to consider an API that models a playlist that consists of multiple tracks, each a sequence of clips -- more complex models are possible.

nigelmegitt commented 4 years ago

@palemieux unless I've misunderstood it, although I agree it would work, the current API is somewhat inefficient when it comes to your first case in https://github.com/WICG/video-editing/issues/13#issuecomment-638426885, because the concat operation takes a sequence<MediaBlob> rather than the putative result of a not-yet-finalised MediaBlobOperation.

This means that in order to evaluate the concat with the API as it stands, a finalise operation must be completed on each of the 3 implied trim operations so that their resulting MediaBlobs can be concatenated.

This is not only a possible problem with efficiency, but it also introduces the possibility that unnecessary transcodes could take place, which are avoidable if the processor knows the desired output encoding. The effect of those transcodes may be to reduce image quality, as well as taking additional processing time.

sroucheray commented 3 years ago

Following @palemieux's comment I have a naive question. While playlists cannot model multiple overlapping tracks, EDL can, It is widely used and for a very long time in the broadcast industry. Furthermore it can be a good interchange format with desktop editing systems https://en.wikipedia.org/wiki/Edit_decision_list.

Wouldn't a WebEDL format and API be a good candidate for this proposal ?