Processing Improvement Discussion (ImprovedProcessinf branch)

[ilia3101]

Originally this issue was me asking a question about compiling speed, but it has become a discussion about implementing many features in processing. So some of the comments at the start are irrelevant.

Original first post...

---

Never ever had it compile without it, however I do end up with a 'mlvapp' execultable that I run.

Don't know what this means:

tar -C /home/ilia/bin -xvJf /home/ilia/MLV-App/platform/qt/FFmpeg/ffmpegLinux.tar.xz --strip=1 ffmpeg-4.0-64bit-static/ffmpeg
tar: /home/ilia/bin: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
make: *** [Makefile:481: mlvapp] Error 2

[dannephoto]

Holymoly. I missed the color party completely. Nice progress!

[masc4ii]

@ilia3101 : I found the reason for the flat pictures: it has to do with "Allow Creative Adjustments". If I never grey out the checkbox, I just have to enable it, and the pictures aren't flat anymore. In a commit you delete "use_rgb_curves"... this enabled e.g. "curves" before - this decision isn't there anymore, just allow_creative_adjustment is still there. So e.g. the curve only works, if the checkbox is set.

I have a question: when do we grey out Allow Creative Adjustment, and when we don't? Especially in "Custom" it makes not always sense to have it enabled...

[ilia3101]

Ah yes, sounds like a correct explanation. I want to have all creative adjustments enabled/disabled just by one variable (allow_creative_adjustment), including curves. I'm sure it's just a matter of changing a couple of if statements, but I'm not sure where to do it. Do you think you could fix this easily?

[ilia3101]

I see you have added special tonemapping functions for very accurate rec709 and sRGB gammas, is it ok if I don't include them for now and just do it as a simple output gamma value for those profiles?

I promise to bring those accurate functions back later. This is just to make it a little simpler for now.

[ilia3101]

I have a question: when do we grey out Allow Creative Adjustment, and when we don't? Especially in "Custom" it makes not always sense to have it enabled...

Ah good question, I think we should just not grey it out anymore. This means it can also be useful to compare how the image looks before and after.

[ilia3101]

@masc4ii Is it physically possible to transfer your rbf denoising to work inside apply_processing_object? Right before the final colour space conversion is done. I think it would create a better colour denoising result if it's done in the processing gamut.

[masc4ii]

Okay, thanks for the info. I can try to do this. I just needed the info how to do.

rec709 and srgb doesn't matter if it is not there. What I (and I think also others) need, is BMD-Film profile support, because there are many LUTs out there bringing a very nice looking picture. I would really miss it.

All the RBF functions have to be rendered on full picture, because the enemy's name is "boarder". So you'll get very ugly stripes and artifacts when doing this in the threaded function. Maybe we need to build the same thread mechanism after denoising and grain, for post conversion? Or you use OMP there, this solution is mostly very very simple, because you just need one single line with #pragma ...

[ilia3101]

What I (and I think also others) need, is BMD-Film profile support, because there are many LUTs out there bringing a very nice looking picture. I would really miss it.

Sure, it will stay.

All the RBF functions have to be rendered on full picture, because the enemy's name is "boarder". So you'll get very ugly stripes and artifacts when doing this in the threaded function.

I see. Could split it in to two halves and apply it in between, but let's leave it for now.

also

I've been thinking of just using rec709 as the processing gamut (after doing tonemapping in a wider one to avoid the ugly clipping). Rec709 sounds limited, but if the lookup tables represented a range of values: something like -1.0 to 3.0, instead of the current 0.0-1.0, the gamut wouldn't be limited to rec709 at all, and would allow final output in wider gamuts. This may increase banding but it's still a really small risk.

I just think the rec709 gamut is very good for pleasing processing results and is why MLV App produced especially nice results before.

[bouncyball-git]

but if the lookup tables represented a range of values: something like -1.0 to 3.0, instead of the current 0.0-1.0, the gamut wouldn't be limited to rec709 at all, and would allow final output in wider gamuts.

I just think the rec709 gamut is very good for pleasing processing results and is why MLV App produced especially nice results before.

Hmmm... very interesting approach.

[masc4ii]

Could you please try out what we have now? GUI should work as expected now. Missing: Gamma and Tonemapping Function. Strange: Processing Gamut is Rec.2020 if "Tonemapped" is selected. If I manually select Rec.709 it looks like in old versions. Is that correct? I would expect, Rec.709 is chosen automatically. @Ilia: have you seen the comment in the ML forum thread about ACES?

[ilia3101]

Thanks for the interface update, interesting to try out gamuts so easily. Will have more of the parameters setting functions soon.

[ilia3101]

@masc4ii Seems like there will have to be a "lighten" type parameter added to the processing profile section, it is the only way I can see to replicate the tonemapped profiles correctly, as the old way of "3.15 gamma" was very bad colour science grammar.

[ilia3101]

Can we add a mouse over explanation pop up to some of these parameters? I feel like it might end up being a confusing section.

[ilia3101]

Strange: Processing Gamut is Rec.2020 if "Tonemapped" is selected. If I manually select Rec.709 it looks like in old versions. Is that correct? I would expect, Rec.709 is chosen automatically. @Ilia: have you seen the comment in the ML forum thread about ACES?

Very correct! I am still deciding what the default processing gamut should be, just put Rec2020 randomly for now. And setting it to rec709 should produce old results, so it is good that it does. I want to use something wider by default when we actually release, as it will work better for low light (so Danne uncoloursciecence fix can be removed).

I know changing processing gamut right now causes HuGE changes in the output, uglier colour mostly, this will be fixed and should not be happening (only small changes should happen).

[bouncyball-git]

Tested with Danne's bluish clip with temp = 3000K:

Rec709/sRGB processing

Rec2020

aces ap0

ACES processed version looks less bluish. Multiplier function of WB code really needs to be revised right? @ilia3101

[dannephoto]

Yes, glad you brought this up. I also tested with another clip and same wb issue. Really nice to see these color gamuts being implemented. Now back to the wb issue. I am so frustrated to not knowing at all where to look for hiw to fix it. It kind of puzzles me how colors seems right but at the same time they crack into blue in certain conditions.

[bouncyball-git]

Yeah color processing is very interesting bur IS another world. And needs so much dedication and time to comprehend it to usable degree to jump into Ilia's and Masc's processing development :). I'm off right now... no time at all.

Guys thank you for your time and goodies you are producing!

[masc4ii]

Can we add a mouse over explanation pop up to some of these parameters? I feel like it might end up being a confusing section.

Sure. Tell me what phrase you would like on which element. Or, go into QtCreator, search elements in UI-Editor and write it. 😄

I added the tonemap function now to the GUI. All LOG options seems to be very bright... but I think it is because gamma is not working yet. If selecting a profile, "getTonemappingFunction" seems not to bring the correct value. It is always "None", but looks like "Reinhard".

Edit: I think it is, because in processingSetCustomImageProfile(), processing->tonemapping_function is not set.

[masc4ii]

Hi @ilia3101 , any progress on your side?

[ilia3101]

Hi sorry, gone on holiday. I was going to try spend some time on this, but I broke my 5D with a unlucky fall and not feeling much motivation, not until I find another ML camera at least. Might ask on ML forum where I can find an exceptionally cheap 5D mark II.

@bouncyball-git thanks for doing that test, it really doesn't look great :(

@masc4ii or @dannephoto could any of you test that clip in Adobe software to see how that handles it?

[ilia3101]

Yeah, quite a few options are not working yet so I imagine log might not

[ilia3101]

And I know the results in this branch still don't look right, they reallly don't especially with red colours.

[masc4ii]

Oh noooo... not again! It broke in holiday? F***. Can understand you are not very motivated now. But it is a shame for this topic. I had hope, we solve it this time... Wish you much luck to find another 5D2 or 5D3. I think with other MLV camera's, you won't get happy. E.g. EOS M is a nice little toy with many great features, but quality from 5D2 is still another class.

To just have a matrice multiplication to get from one space into another is right, there is no offset needed? (for having the center (D65) always at the correct position) There are combinations between processing and output gamut, where all becomes red or blue... right.

I could compare in Resolve. I have no idea how to do in Lightroom (is it possible here?).

[bouncyball-git]

I could compare in Resolve. I have no idea how to do in Lightroom (is it possible here?).

I think he meant develop just one DNG frame from that Dann's clip in adobe product. Temp should be 3000K or less.

[bouncyball-git]

I broke my 5D with a unlucky fall and not feeling much motivation, not until I find another ML camera at least. Might ask on ML forum where I can find an exceptionally cheap 5D mark II.

Uhhh... not again, NOT AGAIN!!! 😁 (your sensor was not calibrated properly anyway). Get 5D3 and you'll be set up for cuople of years from now.

[ilia3101]

Oh noooo... not again! It broke in holiday? F***. Can understand you are not very motivated now. But it is a shame for this topic. I had hope, we solve it this time... Wish you much luck to find another 5D2 or 5D3. I think with other MLV camera's, you won't get happy. E.g. EOS M is a nice little toy with many great features, but quality from 5D2 is still another class.

Useful to know that 5D2 is much better than EOS M, I was considering EOS M but I won't be getting it as main camera then. I will keep looking for a 5D2 or 3. There's something satisfying about using a 10 year old camera, 5D2. 5D3 would still be better of course.

To just have a matrice multiplication to get from one space into another is right, there is no offset needed? (for having the center (D65) always at the correct position) There are combinations between processing and output gamut, where all becomes red or blue... right.

This is true, colours might shift when adding saturation in the gamuts with D50 white point and ACES with D60. It might not be very noticeable, but could be compensated for by using D65 as saturation mid point instead of the processing gamut's default white/grey. (can't explain too well)

I could compare in Resolve. I have no idea how to do in Lightroom (is it possible here?).

What bouncyball said ^

[dannephoto]

What! Not having the eosm as a first cam ;). Too bad your 5D2 got broken. Sad to hear. I did a few tests and had a few ideas too. White balance is still problematic. I can´t fix it. Too complicated but in time I think it will be fixed.

Check following pictures downloadable here including MLV and 3D lut: https://bitbucket.org/Dannephoto/mlv_app_compiler/downloads/test_aces.zip

Original

iso_2300

03_wb_2300_unscience_color_fix

04_aces_ap0_rec2020(colors wonky)

05_Aces_ap0_Aces_ap0(double dose ap0. We need passthrough)

06_Aces_ap0_Aces_ap0_lut aces_to_rec709(with passthough instead of double dose aces ap0 the added 3D lut should look much better)

I wonder if we should have "passthrough" options in the color space rolldown window? If we select aces gamut in first we migh wanna keep that exact setting on export and use 3D luts in post. Also. If we select ap0 in first roll down and rec2020 in output, colors should be perfect coming back or am I wrong here?

Oh and adobe camera raw iso 2300(working good):

[dannephoto]

I wonder if we should have "passthrough" options in the color space rolldown window? If we select aces gamut in first we migh wanna keep that exact setting on export and use 3D luts in post. Also. If we select ap0 in first roll down and rec2020 in output, colors should be perfect coming back or am I wrong here?

EDIT: Answering myself. Yes rec2020 don´t specify color gamut. But how do we specify color gamut on top of aces if that was selected? Should we have the aces_to_rec709 gamut or aces_to_alexa_wide gamut etc in the second rolldown window? A two-step operation. First establish color gamut, then add output stuff?

[masc4ii]

Passthrough: When looking into the code, I find this:

/* Final colour space transform */
    if (processing->output_gamut != processing->processing_gamut)
{
...
}

So if it is equal, there is no conversion in the end of processing.

[ilia3101]

Yes there's just pass through if the same colour spaces are selected. I think the wonky colour issues are because I haven't yet sorted out processing to make it linear at the final conversion stage (you'll notice i do a stupid gamma thing there that's slow and bad and not too accurate)

[ilia3101]

@dannephoto thanks for the adobe comparison. Looks smoother, but it is both colder and less saturated. Could you try and match the colours with MLV app a little closer? Using saturation and whitebalance in whatever way required, I just want to see how it rolls of those highlights in the background.

[ilia3101]

Off topic, but the MLV App tonemapping causes a bit too much flattening (visible on your comparison with ACR), and I have noticed it makes me grade shots darker than I want often. I will add another reinhard function but affecting a smaller range at the top.

[dannephoto]

Aha, I see. Double selection same as passthrough? No time atm checking adobe stuff but other than wb breaking in colder kelvin regions mlv app works just fine.

[ilia3101]

I must take a complete development break from MLV App until late June for educational purposes. I do want to continue this branch later of course. It is important. Though if I get a camera (even eos m) earlier than June, I may be partially back before June (can't resist).

Main question:

Is this branch at risk of becoming unmergeable with the main any time soon?

[masc4ii]

Take your time Ilia. If it will be mergeable... I don't know. At the moment it still should be. I do not plan any bigger new features or changes at the moment, so for now it should be safe.

[ilia3101]

Same for the better blur for Highlights/Shadows. You can switch it in the code only atm. I use it already and I am very satisfied. Maybe it is also okay to just use the better blur... what do you think?

Have you put he better blur in yet as defautl? I think it is a good idea to just have it as a default, no switch. It will only improve all old edited files if anything. I think you should do that and it can be release time. That is enough of a reason to release.

[ilia3101]

I've been thinking of just using rec709 as the processing gamut (after doing tonemapping in a wider one to avoid the ugly clipping). Rec709 sounds limited, but if the lookup tables represented a range of values: something like -1.0 to 3.0, instead of the current 0.0-1.0, the gamut wouldn't be limited to rec709 at all, and would allow final output in wider gamuts. This may increase banding but it's still a really small risk.

I will do this. Just quoting to remember. There will be three selectable gamuts: a initial tonemapping/exposure gamut, processing gamut, rec709 by default (with extended dynamic range like -1 to 3), and a output gamut. If all three are the same it will just pass through with no conversion

[masc4ii]

Have you put he better blur in yet as defautl?

Yes I have.

That is enough of a reason to release.

Really? Haha... okay.

[ilia3101]

Have you put he better blur in yet as defautl?

Yes I have.

Yay

That is enough of a reason to release.

Really? Haha... okay.

I think, unless you have more stuff to add. I said that because someone on ML forum said something about leaving camera raw after they saw the comparison between old blur and this.

[masc4ii]

I prepared a release entry. Have a look. Some day I would like to have a chromatic aberation correction - but I haven't found a good solution yet... if you watch my latest youtobe vid carefully, you'll see very bad CAs produced by the speedbooster. But I think I won't find a solution until v1.7.

[dannephoto]

Speed booze! Can you link? Off topic: Trying to compile mlv app for windows. Anybody succeded with mingW on mac?

[masc4ii]

@bouncyball-git tried on Linux with success. Win64 static version is build this way, if I understood right. https://www.youtube.com/watch?v=9cH8ZO-DbUo See e.g. first scene near the edges. Speed booze... LOL.

[dannephoto]

Nice! Sound of music 2.0 :). Will check later on my lap top.

[ilia3101]

Chromatic aberration is an interesting problem

I haven't seen many (or even any) open source programs with good solutions.

Does davinci resolve even have it?

[bouncyball-git]

Great VID Markus!

Hmm,,, I did not know there is some sort of speedbooster for canon EF-M? Do you have Viltrox?

[masc4ii]

@ilia3101 : I am not sure if the new Resolve beta has it - I think I read something about, but I am not sure if this feature is free.

I found this repos, but haven't tried yet: https://github.com/vicrucann/corrCA-prototype And this paper looked interesting too, no idea if we get something like that implemented: https://www.researchgate.net/publication/221159248_Chromatic_aberration_reduction_through_optical_feature_modeling Maybe a starting point...

@bouncyball-git : thanks 😄 Yes, it is the Viltrox Speedbooster + EOS M + EF 16-35 2.8 II.

[bouncyball-git]

Win64 static version is build this way, if I understood right.

Yes that's right. Compiled with mingw64 under Linux and linked with statically compiled QT/etc libs.

[dannephoto]

Trying to get it working under mac. Started out yesterday and now I am waiting for mxe to compile dependencies. Only been 6 hours wait so far. Crap!

[bouncyball-git]

Yup MXE is the way to go :). QT compiling is extremely annoying for sure hahaha... that is why I'm using 5.9.1 (compiled long time ago) all the time. At least it is working w/o probs so far.

[ilia3101]

Win64 static version is build this way, if I understood right.

Yes that's right. Compiled with mingw64 under Linux and linked with statically compiled QT/etc libs.

impressive