[Feature Request] No picture lock interchange

[alcomposer]

The normal workflow for video editing is to (with VFX steps omitted):

1) Edit video 2) Picture lock 3) Export audio for editing in DAW 4) Color grade

This is a legacy workflow, imposed onto current tech for many reasons. However there is no reason why Video Editing & Audio Editing can not happen at the same time, even in different software.

Such a solution exists: http://vordio.net/ It allows interchange between NLE & Reaper DAW.

It would be exciting in the future to allow such an interchange to exist between Ardour and Olive. (for example). As they are both FLOSS projects the file formats of each are freely available to examine.

Such a workflow could be:

1) Edit in Olive 2) Export Olive edit as Ardour DAW project 3) Edit audio in Ardour DAW 4) Re-import from Ardour DAW in Olive

[DaniSeeh]

Oooooh, I would love this with Audacity project files. Especially if you could import the Audacity project file into Olive and right click relink to update it whenever changes were made

[alcomposer]

Audacity is a great project, but compared to Ardour is not really up to the task of dealing with huge Audio post. Ardour one of, (if not the only) FLOSS project closest to ProTools.

[sobotka]

Duplicate of #310.

[alcomposer]

@sobotka I didn't realise that OpenTimelineIO was intended to be used back and forward between DAW & NLE. Makes #310 much more exciting!

[sobotka]

@alcomposer You'll probably see that in all discussions I get involved in from editorial to compositing, I bring up interchange as a core design tenet. Kitchen sink applications simply don't quite get the job done, and playing well with others is prudent to fit into a pipeline. Interchange plays a key role in:

• Music / scoring.
• Foley / sound effects work.
• Compositing.
• Grading.

TL;DR: It's pretty darn important. OpenTimelineIO is one of the more interesting projects to appear in a long while in this regard.

[frink]

@sobotka - I'm very much of the same mind. But more than simple interchange, we really need to think carefully about workflow. The hard part is getting people to agree on such a universal interchange format...

Otherwise: https://xkcd.com/927/

[sobotka]

The hard part is getting people to agree on such a universal interchange format...

Perhaps you didn’t read the documents around OTIO?

[frink]

Correct. Have not read the document. Of course, I was referring to the open source world. Getting Ardour, Olive, Blender, Natron, and Kdenlive on the same page may prove difficult...

[sobotka]

Can only control what can be controlled. Plenty of non dumpster fire applications support interchange.

[frink]

When we decide on and start implementing a good interchange format Iĺl talk to Paul about Ardour support. Wee had several long conversations about features and most of my requests have been implemented...

[alcomposer]

I think it would be a great idea to get OpenTimelineIO in Ardour & Olive. Would enable a very powerful interchange workflow, and lessen Olives need to be a kitchen-sink app.

(however for my workflow- using second system sound, I would still need sub-frame sample accurate audio in Olive)

[frink]

In order to do an interchange with Ardour you would need that anyway. The underlying Jack Audio is extremely precises to a fault. Xjadeo is the first attempt by Ardour to start to address sub-frame accuracy. It works well if you just have a single video clip to sync to...

Great start, but audio for movies is much more complex. I believe that full bilateral interchange within a geographically distributed team is the direction that we should shoot for. This means more than just a simple interchange format. We also need approval workflows and network project exchange protocols that make sense for the decentralized modern world. This is the pro-level stuff that no open source NLE has been interested in solving. But even commercial software is facing the same problem.

We can't truly solve the format interchange problem without understanding the management and approval workflow problems. I don't know if this is something that should necessarily be done inside the NLE but I can see several good arguments for it. A director making notes from dailies in various stages and then sending them off to the other side of the world.

VFX houses are the current adopters to FLOSS solutions in this industry because they are trying to milk so much out of their hardware. They are also the ones who need the most back and forth between their work and the director. There is a lot of really good information detailing this problem in the special features for Lord of the Rings. While the industry technology has grown a bit since then, Its more of an issue now than it was then. As files get bigger everything becomes more complex in media.

I mention these things because I think it's just as much part of the discuss on a truly great interchange format. Here is my current spec on what an good interchange format should include.

1. have sub-sample accuracy in alignment
2. Contain both audio and video information
3. Maintain references to original files when compiled media used (i.e. original audio files containing Foley and music and ADR etc...)
4. Allow for exchanges in edits of both time alignment and processing of the media files
5. Allow, require, and inforce versioning and approval of new versions
6. Allow for partial synchronization of timelines when full synchronization would be too bandwidth intensive
7. Allow for sub module like projects to avoid monolithic repository problems (think SVN/CVS)
8. Avoid needless copy of media files when they travel between different programs

These are the things I can think of from the top of my head. As you can see, a time locked interchange is only the tip of the iceberg. But approval and network interchange are just as crucial to modern studio workflows...

[sobotka]

Great start, but audio for movies is much more complex. I believe that full bilateral interchange within a geographically distributed team is the direction that we should shoot for. This means more than just a simple interchange format. We also need approval workflows and network project exchange protocols that make sense for the decentralized modern world. This is the pro-level stuff that no open source NLE has been interested in solving. But even commercial software is facing the same problem.

I think this sort of is over-inflating the challenges here a bit. Interchange is less required for scoring in some instances. Most of the need for scoring and audio is addressed by timecode support I believe.

If you have a timecode stamped edit, you can score it in most applications.

If you have interchange, you can reconstruct the timings required on your assets in your tool of choice, although again, this is more than likely covered by timecode support.

It really is that simple, or at least a small and achievable goal for Olive, without burying development.

That alone makes it feasible for a small independent person or small team to score their piece, post produce the effects, do sound design, etc.

This obviously interlocks with the need for SMPTE timecode, as mentioned in #628.

• An excellent summary of the software needs for scoring a film from Spitfire.
• A decent little note from a composer on Reddit.
• An answer at the Apple forums on the importance of SMPTE timecode in scoring.

[frink]

@sobotka - You're right. I am over inflating the issue to some extent. But there are increasingly more cases where you need interchange. In my comments, I wasn't so much thinking scoring as I was Foley. Essentially, the Foley needs to be locked to the underlying video - sometimes before compositing is completed. Particularly with CGI defining more of the screen. Often the Foley audio is part of the cues used for editing the video.

But you are right that SMPTE works well for the classic workflow. Especially for scoring!

Shoot -> Import -> Sequence -> Composite -> Color -> ADR -> Foley -> Score -> Sync -> Mix -> Master

My concern is that a distributed teams brings a new set of rules to everything. While the workflow above is still preferred, I'm seeing more people breaking the rules on this every day.

Another case where bilateral sharing needs to happen is in the case of music videos.In another thread I mention the need to have tempo maps exported as a snap grid for music videos. If you've ever watched someone try to sync video with audio in Avid or Premiere - it's painful to do complex edits. SMPTE does not solve this problem since we're talking about musical time divisions instead of standard timelines.

Another case were exporting changing timelines is important is with CGI composited characters. There are basically two ways to do this. The old way was to CGI the characters and do compositing and then cut the video. But increasingly, there is a back and forth between composition and editing to get the sequence right. You need both timelines to be flexible to the changes of the other. (Thus the close relationship between Premiere and After Effect for example...)

Finally, color grading is becoming so common that it is done earlier in the process and refined as time goes on. This timeline also needs to be flexible to the interchange between editing and coloring. Thankfully many modern NLEs have included coloring as a first rate citizen for their process and keep it in the editor synced to the edit timeline.

There are many ways to solve this problem. However, the most flexible is allowing the Unix philosophy to reign and have different programs (and possibly people) do each part of the process. I believe that if we can initiate this sort of collaborative standards between existing open source programs we can produce something that is better than the commercial offerings.

To be clear, I wouldn't want to burden development. Instead what I propose is finding an open standard (OpenTimelineIO seems to be the best suggestion...) and then writing to and from that standard so that interchange can happen between various programs.

Ideally, we also need approval models that are either included in this format or in greater file management.

Should the NLE do every bit of this colossal task? Probably not. But it does need to understand the ecosystem that it exists to support...

Hope that clarifies. 😄

[itsmattkc]

I shall merge this into #310 for now, but I agree that it's something to strongly consider in the core design.

[frink]

We need to decide who compositing and coloring fit into the Olive ecosystem. Ardour seems to be the main focus for serious audio. Ardour has a tempo map rewrite in progress. Once Paul releases that we may be able to tie into that for easily doing the advanced tempo stuff I mentioned earlier...