[COLOR] Color Grading

[frink]

With Davinci, Lightworks, HitFilms, Blender, Kdenlive, Shotcut and Openshot all offering various levels of color grading as part of their editing experience any modern pro editor will be considered incomplete without such facilities. Building a GLSL we may be able to create much more of a seamless experience that the prosumer editors. If we want to go head to head with the professional editors we will need to add motion tracking which is a much bigger discussion.

While we already have most of the facilities of great color editing we lack some of the interfaces that are familiar to work with these things. These include some of the graphs and also the color wheels that everyone is used to. As we continue down the path of a pro editor, color grading is a must in order for Olive to be feature complete...

[ghost]

Color grading support is coming according to #415

If you have the time, I would recommend reading issue #237. It should explain why color grading is not in Olive yet.

[frink]

Will read thanks!

[frink]

This is nearly a duplicate of #415 except that I think it needs to go further. I'm not sure if the NLE is even the right place for color grading or if we need a standalone coloring editor. Something like Darktable with object tracking would be idea and actually resemble more of the way Davinci Resolve does coloring - which is absolutely stunning. However, even layers above with different shapes, merge styles and opacity coefficients would go a long way towards complete color manipulation.

I agree with everything said in #237 - this is not an easy thing to get right. Curves are a good place to start, but are only really useful if you can break up the image into various regions that have different curves rather than trying to treat the whole thing monolithically. Of course I realize that statement just complexifies the request on a factor of about 30x.

It would be ideal if we could have a separate clip editor that would deal with coloration of one or several clips. This is a vast subject that is in need of much more detail than any open source NLE has considered to date. Kdenlive has done a decent job at trying to handle it but really it note-base compositors like Blender and Natron that actually have a chance of getting it right. However, I don't think that we should have to run things through composition nodes in order to arrive at an emotional color palette. I think that's why so many are in love with Davinci Resolve...

[Corruptinator]

Not going to lie, Davinci Resolve is a pretty nifty software, problem for me though is that some computers I use are not that powerful even if I have the top of the line model to make the computer optimal.

[frink]

Agreed. Also the Linux support is troublesome. The built on a lot of cutting edge hardware ABIs that are still not completely standardized...

[frink]

But it would be great to have their power window features in a simple open source editor. Can be done with Blender but it's like pulling teeth to get there. Kdenlive can kindof do some of that. Natron can handle complex grading but at the expense of running a very resource hungry compositor. In theory GSLS should not make these things overly process hungry if coded properly. But theory and practice are often miles apart.

[frink]

If Darktable or Rawtherapee did video then I'd suggest linking to them instead. But the key frames involved and whatnot... I don't think this is likely... But that might be an interesting way to separate coloring from scene editing...

[GlennWilk]

I know it may have been mentioned in other posts, but I frequently use LUTS and I would love to see that feature implemented. Thanks

[frink]

@GlennWilk - What is your process for generating LUTs?

Perhaps using the digital developers tools (DarkTable or RawTherapee) for LUT creation or even GIMP or Krita for that matter would be agreeable. The biggest question is how we should store the LUT.

As far as color grading they are about the most simple and profound way to work. OBS uses LUTs effectively in real time streaming. It's QT and GPL v2.0 using QT so we might be able to just grab their implementation and plop it in... I've not looked at the code personally, but the basic concept of a LUT is so easy that it should be trivial to include.

Ideally, if we could generate LUTs based on keyframing various sliders and then use that 3D LUT as a reference for coloring we should be able to allow changes in color over time.

Alternately, we could force the use of static 2D LUTs but use some form of vector morphing to apply the color over the file. Using vectors should allow arbitrary precision when using proxy videos or full size sources.

[frink]

More on the idea of moving vectors for LUT windows. Davinci Does something like this with its power windows functions. One might be able to use similar techniques as what is used to animate SVG movements for changing the keyframes around.

SVG is a simple enough file format that we will want to support anyway.

Once we have OpenCV integrated we can plug in their point tracking and get the full shebang in terms of moving color grading around...

As I think about it, it makes since to have moving areas that define the LUT using a static 2D LUT. The only thing that you may want to add is an interpolation multiplier to determine how much of the LUT to apply and then also a feather amount which would effectively be calculating a non-fixed value for the multiple.

You might also look at the layer blend models that OBS uses for it's layers. We need decide whether the LUT approach or calculated blend model is better. My guess is that LUT will be the preferred way to go as long as there is an application multiple and a feathering amount.

[GlennWilk]

I have purchased a set of 3D Luts and use them with software like Shotcut or Photoshop.

[frink]

LUT import/apply is probably a good place to start. But we need to be able to generate LUTs dynamically in order to fully control things.

Anyhow, I like the idea of using LUTs because the implementation is so simple...

[ghost]

I made a simple (sort of) color grading effect for Olive and I posted it here. I hope it can help :)

[CodigoCristo]

kdenlive reference LUT file... Interpolation Mode...

No Lut

yes Lut

File CLUT

[AlexDaniel]

Just to be clear, I'm here because every year I try to find software to do basic video manipulation and color grading for my personal needs. And every time I'm very disappointed, but I think olive is the only candidate to get it right (besides davinci resolve, which already has it right, but I currently cannot run it on an AMD GPU, otherwise I would've been using it).

I'll write some things that are perhaps obvious, but to developers of all other open-source video editors they're actually not, for some reason…

So, some background info first! Any primary color operation can be represented by a LUT. In fact, any number of color operations can be represented by a single LUT. Moreover, a LUT can be applied on the GPU in real time with very minimal overhead. Basics about LUTs are described here.

Moreover, any operation done on the LUT itself will give you a new LUT that achieves the same effect. This means that if you're implementing let's say RGB curves, you can do all of the math on the CPU (which is extremely easy to implement), get the resulting LUT, and then apply it to the video on the GPU.

That's what Davinci Resolve did right and that's why their software is the only one that doesn't absolutely suck:

• Kdenlive has separate filters for every effect, so doing more operations means more overhead and worse performance. I think they're doing everything on the CPU too, but I'm not sure.
• Blender is doing everything on the CPU. It's mind-blowing that software which deals with GPU rendering is doing compositor effects on the CPU (basically just looping over all pixels). That's why their compositor is not real-time, and you can't use it for video even if you really wanted to.
• And everything else simply doesn't have any tools for color grading. Shotcut comes with color wheels but no curves, and so on.

Now, in davinci resolve, all color grading nodes are homogeneous. Any color grading node can be used to achieve any color-related operation. You decide which node will be for saturation, which node will be for curves, and which node needs more than one effect at the same time. Why? Because it doesn't matter, it's all just LUTs underneath. It simply generates LUTs for all nodes (possibly even combining multiple nodes into a single LUT) and applies it all in real time. So simple yet so good.

I think there's a suggestion somewhere in this thread that maybe it should be achieved with external software. The answer to that is NO, because you cannot do color grading blindly through the export-import cycle. There are also masks and keying. Masks are just texture multipliers – just like you can decide to use a value between 0 to 1 to apply a LUT (where 1 means use the lut fully and 0 means no effect at all), masks are exactly that but they're textures, meaning every pixel has a value. That's very efficient on the GPU too. Keying is a bit more tricky though, because on each frame you need a new LUT, but I believe that even in davinci resolve it's a bit slower, so I wouldn't worry about it too much.

So, what's needed in Olive to make it the only reasonable alternative to davinci resolve (compared to kdenlive, shotcut, etc.)?

• A single node called “color grading”
• The node should maintain an internal LUT
• Passing the texture through that node should apply the LUT on the GPU
• Changing any of the controls should change the LUT
• Then just implement all of the basic controls (in that single node):
  • Some basic sliders: saturation, color temperature… actually, those two are enough for me
  • 3 color wheels
  • RGB & Luma curves
  • More curves: Hue vs Hue, Hue vs Sat, Hue vs Lum, Lum vs Sat, Sat vs Sat, Sat vs Lum ← that's it, Davinci Resolve doesn't provide more, and arguably you don't need more

And that's it! With this you can achieve A LOT more than any other existing open source tool. This will satisfy all of my current needs.

The next step is masks, but it's not as useful as you might think. Typically, masks (or qualifiers) work by selecting parts of the image based on color, but that is not as good as it sounds. Any video has noise, so your masks will be noisy too. Even in davinci resolve, abusing masks will result in very noticeable artifacts on the video where certain spots go in and out because of the noise. In general, the recommendation is to use primary wheels, curves, etc. as much as possible, because those change the video uniformly.