Processing Improvement Discussion (ImprovedProcessinf branch)

[ilia3101]

Originally this issue was me asking a question about compiling speed, but it has become a discussion about implementing many features in processing. So some of the comments at the start are irrelevant.

Original first post...

---

Never ever had it compile without it, however I do end up with a 'mlvapp' execultable that I run.

Don't know what this means:

tar -C /home/ilia/bin -xvJf /home/ilia/MLV-App/platform/qt/FFmpeg/ffmpegLinux.tar.xz --strip=1 ffmpeg-4.0-64bit-static/ffmpeg
tar: /home/ilia/bin: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
make: *** [Makefile:481: mlvapp] Error 2

[ilia3101]

Also is there any chance of making it compile faster? Takes so long even with -j4 on my 'new' i5 5200u dell :cry:

[masc4ii]

Puh... no idea. Have you tried checking out - compiling from the start? I guess yes. The lines you posted are for ffmpeg binary. Have a look if it is there and if it is copied.

Yes, we could make it faster. For me the main problem for slow compilation is MainWindow.cpp. This file is soooo huge. But I don't know how to split it in small packages. There are so many functions which need access here and there. 😞

[ilia3101]

Ah does ffmpeg need to be extracted first?

Don't make faster compiling a priority don't worry for now.

[ilia3101]

Can we do a quick release in approximately 2 days

[ilia3101]

@masc4ii do you know why it might be crashing every time I press the open mlv or open session? Even without my changes... no file picker window shows.

[masc4ii]

The command in the .pro file should extract ffmpeg automatically, if I remember right.

We can realease whenever you like. Do you plan something special? Atm I plan integrating a grain (noise) generator and maybe a switch for the different blur algorithm for better Shadows/Highlights.

Those crashes I had a long time ago. But I don't remember the reason... sounds like a problem with the libraries. Under (left bar) Project, (chose project) Run, there are some checkboxes with DYLD_LIBRARY_PATH... maybe try playing around there.

[ilia3101]

I would like to do a release on April 1 as it is an important day in magic lantern and I am going on holiday on April 4 so it seemed like a suitable day. Hopefully a couple of improvements I'm doing will be ready by tomorrow or on the 1st.

And I will try installing qt creator and doing what you say, just using terminal right now.

[masc4ii]

Oh... terminal?! OMG... never tried that 😄 Let's see if I can get ready my things too...

[ilia3101]

I am in a new branch right now, BetterProcessing. What kind of things are you planning, processing?

I have been using terminal almost the whole time :rofl:

[ilia3101]

And it doesn't crash when I run from Qt creator arghhhhhhhhH!!!!!!!!!!!!!!!1 :(((

But when i do ./mlvapp it crashes again :(

[masc4ii]

Japp... as expected. Library problem.

Yes, the noise generator is in code already, but there is no parameter yet to set it up. Same for the better blur for Highlights/Shadows. You can switch it in the code only atm. I use it already and I am very satisfied. Maybe it is also okay to just use the better blur... what do you think? What does better processing? ACES?

[ilia3101]

I completely agree, your better blur looked incredible for highlights and shadows, just switch it out instead of my blur already.

The better processing branch is what I've been wanting to do, but not just ACES.

[ilia3101]

Japp... as expected. Library problem.

Do you know how to recreate the way Qt runs it but in terminal? Is it some kind of environment variable that needs to be set? As I don't like switching between two text editors all the time to run it.

[masc4ii]

It might happen if you have different Qt versions installed on your system. If you have just one Qt, you'll (hopefully) never have this problem - at least I know this only from systems where some Qt library was installed systemwide, but you program with a different version.

[ilia3101]

Don't think I have different Qt versions installed. This is confusing. I'll figure it our later, for now I'll just drag files on for testing.

[ilia3101]

@masc4ii Would you be able to add a profile section to the interface? It would need to have a profile selection menu at the top, then some menus and options below it. The lower options would be stuff like output gamut, tonemapping function, gamma etc, and would all change at once when a profile is selected, and when the user changes any of them, the profile will become 'custom' - is this possible? or too much?

Compatibility with old files will be kept as all the same profiles will be available still.

[masc4ii]

Grain-Gen (simple noise) is working now. Yay.

If you like, we could do try that. So the profile (and also Cam-Matrix?) will get a separate section somewhere? And old profile will be splitted to Profile and ...? What should be the options in which combobox? Not sure if we can have compatiblity... this sounds difficult to realize. But we could try also this, if you have an idea how.

[ilia3101]

1 min almost done mockup

[ilia3101]

Would look like this:

The items below the 'Processing profile' option would be set automatically when setting the profile, there will be some getter functions on processingObject for finding their values after setting profile.

Maybe there should be a separator line or something between processing profile and the other options to show that they are kind of like 'sub options'

[masc4ii]

Whohow... this looks like a lot of work you plan on your side. The GUI should be doable... not sure if in one day, but doable.

[ilia3101]

It is a lot of work but not as much as it seems, I have come up with the easiest possible way to do it :wink: (keeping much of the old stuff)

updated mockup:

[ilia3101]

https://github.com/ilia3101/MLV-App/commit/52cb5f9e302f44e4015ab843e8a7eb2268e51568

@masc4ii @dannephoto @bouncyball-git

Can anyone test if the BetterProcessing branch crashes on windows/mac? Only been testing on Linux so far.

And no, it is not yet any use for actual processing so don't use it.

edit: rec709 is the best gamut in the world and all others are terrible :tired_face:

[seescho]

Never ever had it compile without it, however I do end up with a 'mlvapp' execultable that I run.

Don't know what this means:

tar -C /home/ilia/bin -xvJf /home/ilia/MLV-App/platform/qt/FFmpeg/ffmpegLinux.tar.xz --strip=1 ffmpeg-4.0-64bit-static/ffmpeg
tar: /home/ilia/bin: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
make: *** [Makefile:481: mlvapp] Error 2

Do you have "bin"-directiory in your home-dir (Some distributions come without one, I heard)? If no, create one: mkdir ~/bin Edgar

[masc4ii]

GUI elements painted and commited. Next we need all elements inside the comboBoxes. With QtCreator just doubleclick those items and write the texts in the list. Then I'll create the callbacks and we can shot on your macro's/functions.

Edit: can compile without any error on OSX 10.9.

[masc4ii]

I added the most important callbacks for you. Some of them already have the link to processing unit. Anyway, the order of the comboBox entries has to be the same as your defines (if you create them for your functions). Expected: no receipt support at all atm. This means, switching between clips will lead to undefined behavior. I think you should change all combo's after switching a clip to get a defined look.

[ilia3101]

I'll push more code in 1-2 hours when I'm home, it's working but image looks like rawtherapee right now, hopefully can fix that.

[ilia3101]

The log functions should be in tonemapping menu

[ilia3101]

Also processing gamut should have all the same options as output gamut and use setProcessingGamut or somemthing

[ilia3101]

Wait why is Linear an option in gamut? Is it accidentally left over?

[ilia3101]

The log functions should be in tonemapping menu

I will add some macro values and get set function

[ilia3101]

Creative adjustments is really broken, not allowed at all in tonemapped/standard/film mode - not sure why. Is it in the interface?

Sometimes they do affect the image argh what is going on

[ilia3101]

I have a feeling something is really wrong with rec709 it looks so baaaaaaaaaaaaaaaaaaaad, adobeRGB even looks more like old mlv app's rec709 output. WTDFGfkjdsfghksdhfkajdsdfha

[ilia3101]

Well even using AdobeRGB, we can see improvement...

OLD, see clipping on green thing above orange thing:

NEW- no clipping

New looks less saturated as it's AdobeRGB

[masc4ii]

I thought you forgot linear. @seescho will still need it... Okay, so I copy all entries from output gamut to processing gamut.

Creative adjustments is really broken, not allowed at all in tonemapped/standard/film mode - not sure why. Is it in the interface?

If it is greyed out sometimes... yes. If it affects the image, it is in the C backend.

I wrote many lines for receipt support... will commit soon.

[ilia3101]

Well it won't be in gamut, it will be back as gamma set at 1.0 and no tonemapping function

[masc4ii]

Okay. Do we still need "Allow creative adjustments" if we add profile "Custom" ? Isn't this the same?

[ilia3101]

Allow creative adjustments should still be there I think, and if it is disabled, that's when it becomes "Custom" (showing that it's no longer real log)

[masc4ii]

Ah yes... forget about... already saw too much code today... LOL

[masc4ii]

Whoooaaa... the rules for all those comboBoxes will be hard to implement. I get crazy.

[ilia3101]

The way it should work is: The "Processing profile" combobox will call the profile setting function - after this you should call getter functions (such as processingGetOutputGamut processingGetOutputGamma processingGetProcessingGamut) to set the values of the combo boxes and gamma slider and others below it, no special rules in Qt code.

Only rule I think should be to change profile to 'custom' when any of the other combo boxes or settings are changed by the user

[ilia3101]

Or is there something I'm missing that makes it really difficult? I am really sorry if adding saving/copying support will be difficult

[masc4ii]

Not sure yet how it works best without making the interactive rulez between all those boxes to difficult.

[ilia3101]

Setting the "Processing profile" setting will affect the states of the settings below (inside of processing), so I think there should be a updater function called after setting a profile that reads the settings from the processing object and updates them in the interface.

It it's done this way I think it shouldn't require many rules.

[ilia3101]

I will add a processingSetTonemappingFunction(processing, int function) as well as processingGetTonemappingFunction, as well as defines of the functions so the tonemapping combo box can function

[masc4ii]

Okay. I'll try it this way. I expect some fun with the receipts in general... not only MASXML (this should work out of the box if receipts are working).

[ilia3101]

I apologize for using the word function 100000 times in that comment I hope it gets the message across of what I will do

[masc4ii]

I get crazy... if Profiles callback sets the other boxes via getters, their callback functions set Profiles box back to custom... 😢

[masc4ii]

Okay... a part is working now. Looks not too bad. But all clips imported look so flat until I change profile to something else and switch back. Don't understand what happens. Sry.

And some parts... e.g. also the receipt opening calls all element callbacks... very strange things are happening this way.

[ilia3101]

Sorry for my little absense.

Okay... a part is working now. Looks not too bad. But all clips imported look so flat until I change profile to something else and switch back. Don't understand what happens. Sry.

Yeah strange that this is happening.\

... It seems that creative adjustments are not doing anything, must be a problem in processing though as sliders are not greyed out.

[ilia3101]

Sorry I'm really tired. I'll have more tomorrow. April 1st will definitely be missed for having it ready. Also needs some work to get the colours right, as reds look really bad in rec709 and there is huge amounts of noise everywhere.