v1.2 DUAL ISO problems are back..

[gac123]

so saturday i recorded the first show of my friends in dual iso seeing the v1.1 version fixed the dual iso problem, i was quite comfortable doing this

upon opeing mlv app i was pleasantly greeted with the "there's a new version" popup, so i decided to get the new and improved..

~8 hours later the 1 hour show was converted and there are problems every x frames... from a small blue line appearing at the top to (overexposed) highlights not being "un-dual-iso-ed"

see https://imgur.com/a/eXHQPtS

so i re-export with v1.1 (and 8 hours later), no such problems, but (doh!) i forget to export with clog, so olé i'm exporting again for another 8 hours :) (how is the multi threaded thing moving along? :) )

[masc4ii]

Hm... for some dual iso clips highlight reconstruction won't work for sure. With the current reconstruction strategy we can do there absolutely nothing to get it 100% working. Sry to say that. Dual ISO forbids to get it 100%.

[gac123]

oh, i understand things can't be perfect, but this is almost the entire thing.. almost 1,5h footage lol :)

[bouncyball-git]

I never get any color bar on the top. I think it can be windows only (+omp?) thing...

[gac123]

ive also had it in the export

[bouncyball-git]

BTW you had main iso set to 3200 and recovery iso to 800 (Dual iso 3200/800). I'm not sure if it's a correct approach.

[bouncyball-git]

:) we'll see what else can be done.

[bouncyball-git]

Downloaded 1.5gb file. Reproduced very dark frames (DNG Uncompressed).

Edit: It does not matter what kind of DNG or whatewer you exporting. The issue also randomly happens during non frame drop playback.

[bouncyball-git]

Please redownload the win64 build and try to export. Now it is compiled w/o OMP.

[gac123]

oh and the original 10,4gb straight out of camera mlv is done uploading..

yes, 3200/800, normally i shoot 1600 raw, which is the middle ground in what is unrecoverable shadows or hightlights, so for dual iso to be usefull i expanded it a bit

[masc4ii]

@gac123 : please test highlight reconstuction again. I tried finding a better method for getting the currect input value for our algorithm... maybe it works better for you too. Or... could you please upload just this frame you've shown above as MLV? (in the 1.5GB file it is not included, or is it?)

Edit: it can just work, if there is a clipped highlight. If there is none, strange things will happen.

[bouncyball-git]

@gac123

Did you try the last build I gave you? I'm really interested if it works without omp. If it does I have to find out which of 36 places causes this issue.

Regarding iso settings. Usually you set canon iso to 800 and recovery (dual iso second iso) to, say, 1600 or 3200 in your case. Hence there will be no 3200/800 but 800/3200 <= and this is right thing to do.

[gac123]

@masc4ii the frame is included in the 1.5gb file, it's the first frame :) in the original mlv (which is also uploaded.. it's frame 2378)

but here it is with the latest (non omp) version

@bouncyball-git i scrubbed through the footage with the non omp version and idd, the line at the top is no more.. i'm going to export to uncompressed dng again to check

[gac123]

without "camera matrix" setting

[gac123]

@bouncyball-git next show (this saturday) i'll try it the other way around 800/3200

[bouncyball-git]

@bouncyball-git i scrubbed through the footage with the non omp version and idd, the line at the top is no more.. i'm going to export to uncompressed dng again to check

Very nice. Use this latest build with @masc4ii's highlight reconstruction changes then.

[masc4ii]

the frame is included in the 1.5gb file, it's the first frame :) in the original mlv (which is also uploaded.. it's frame 2378)

Okay... then it should be better now for you. I did not see any artifacts there any more with latest commit. Searching now some more clips, where latest commit still fails, to adapt my parameters...

[ilia3101]

Can lowering white level not just fix highlight reconstruction? It always works on normal ISO, is it somehow different on dual?

[bouncyball-git]

Yes it can :(

[gac123]

@bouncyball-git (your) non-omp version looking good! now testing @masc4ii's latest build

[gac123]

i don't see the highlight reconstruction doing much, but at least no issues (will do a full test to confirm)

without reconstruction

what does camera matrix do? (except seemingly boosting saturation)

+reconstruction

[masc4ii]

Can lowering white level not just fix highlight reconstruction?

Nope.

It always works on normal ISO, is it somehow different on dual?

Yes. The information of highest_green (input for reconstruction) gets lost on dual iso stitching, and is different in each frame. In the end it seems like there is a gaussian curve around a peak (which may be there, may be not, sometimes it is small, sometimes it is high). That is why I had to implement an algorithm, which trys to find this peak, if there is one. So I start from full 16bit (65535) and search all local maxima, until I get a maxima which is bigger than a limit. If the maxima become bigger than a smaller limit, I found green signal and there is no peak (->no clipped green). This algorithm has to run for each frame...

Now I am searching for dual iso clips with clipped highlights, to setup the limit parameters to a wide range... But it is very sure that it won't work for 100% of clips. Specially for underexposed clips, noise may look like the highest_green peak... this leads to funny artificial effects. 😛

what does camera matrix do?

See it as a kind of calibrated color for the camera.

[bouncyball-git]

what does camera matrix do? (except seemingly boosting saturation)

It is the future of MLV App ;)

[bouncyball-git]

Without OMP diso processing got 3 times slower :(. But I really do not have time to learn stuff like this now LINK. It is cool to know though...

[gac123]

hmm, i've tried 4 different settings (besides dual iso) nothing, then plus highlight recon, then plus cam matrix, then plus alexa log c all 4 dng's uncompressed exports seem (are) exactly the same

the preview shows the difference of ticking/unticking those boxes but does not follow the export highlight recon settings not working in export¿.zip

also, there are still some problems, this time i notice the highlights on the metal rig of the drums, seemingly full red, green or other pixels

[gac123]

@bouncyball-git yes i've notice the whole thing is much slower, preview, export

[masc4ii]

also, there is still some problems, this time i notice the highlights on the metal rig of the drums, seemingly full red, green or other pixels

Switch of bad pixel fix. Such small highlights are often interpreted as bad pixels.

[gac123]

@masc4ii switching off making it worse :( (more bad pixels)

[masc4ii]

I see exactly 1 bad pixel (on the bass drum) and cleaner highlights now. The boarders of the highlights are a bit too colorful. This might become better when chosing IGV debayer.

[bouncyball-git]

Honestly, I would not use Dual Iso for such a kind of shot. Whole dynamic range have to be spent on stage . You really do not need to see people's backs right?

[gac123]

Honestly, I would not use Dual Iso for such a kind of shot. Whole dynamic range have to be spent on stage . You really do not need to see people's backs right?

well, at a certain point in the show there is a moshpit, wall of death and sitting, so yes, seeing the crowd is actually a big part of those videos, i make these for the band, they can either use some footage for promotional stuff or as a kind of gage to see how the crowd reacts to the setlist

[gac123]

@masc4ii actually, the fix bad pixel option does take away quite a bit of highlights, even highlights on the faces, so not sure how this function works i would assume a bad pixel to be a red, green or blue pixel stuck at full brightness and on its own

[masc4ii]

What this exactly does - I don't know as well. It is copied 1:1 from older/other ML projects, to not re-invent the wheel for each feature. But I know that it has problems on small areas with big contrast difference. So for example on water. And I think, what I saw on your pics should be the same problem.

AMaZE + BadPixel

AMaZE without BadPixel

IGV without BadPixel

[bouncyball-git]

Yup, right. Dead pixel searching code gets fooled by those very contrasty magenta-ish pixels and after interpolating around it often produces even more artifacts.

[bouncyball-git]

Hm... how good looks IGV in this case?!

[masc4ii]

IGV+BadPixels creates dark dots... hard to see, but if you know they are there, you'll see it.

[bouncyball-git]

Tried more experimens with OMP and dual iso and have to say I'm giving up :smile:

Despite it is 3 or 4 times faster now I'm tired chasing all those race conds producing wrong result and random artifacts.

[masc4ii]

Yes... there are 2 possibilities:

1. use OpenMP from the beginning and develop with it -> works and is fast

2. don't use it forever -> works and is slow

3. try to upgrade with OpenMP -> is maybe fast and does often not work

[bouncyball-git]

@masc4ii

Can you give me that 5D2 clip shown above for testing?

[bouncyball-git]

@gac123

Did you test the latest build? I'm so interesting what is the result :)

[gac123]

@bouncyball-git are there new builds? i tested all the once you mentioned in this thread..

BUT I noticed something today!

I made a short clip for something else this time dual iso 800/3200, but wanted a "normal" render time so I tested the last version before omp was disabled, disabled fix bad pixel -> blue or purple line on top of the frame dissapears!

would you please enable omp again with the latest changes (where reconstruct highlight works better, etc..)? I'll try that with and without fix bad pixel to check that top line along the frame

[masc4ii]

Can you give me that 5D2 clip shown above for testing?

You mean my famous boot clip? But it is no dual iso... it was just to explain that bad pixel fix might fail. https://www.dropbox.com/s/igf6ao5cjs1u1m5/short_M23-1205.MLV?dl=0

[bouncyball-git]

@masc4ii: Yeah! Exactly your famous boat clip :) Thanks! I need to check water artifacts time to time.

@gac123: yes this can be done but I do not like the Idea because I'm getting random horizontal stripes in highlights and that's the OMP race case for sure.

[bouncyball-git]

@masc4ii: I think it is amplified 5D2 aliasing on the water. On 5D3 this kind of s..t appears not so often.

[bouncyball-git]

@gac123: latest build with omp

[gac123]

exporting a ~15min clip; 1h45 so far... no line in sight!!

[bouncyball-git]

so far... no line in sight!!

And what about horizontal lines in highlights or black frames?

[gac123]

so the 0h13m45s clip took 1h42m0s

but some bad news (it might actually not matter much), with bad pixel either on or off the preview still is in trouble with that top line.. the highlights seem ok! so please keep omp for now :) when that top line appears there does seem to be an effect over the rest of the frame, a slight boost in the purple color, like a slight purple haze on top as you know it's inconsistent, scrubbing back and forth "fixes" that frame in the preview, see my examples:

for these examples i did boost exposure to make it clear

on the export side of things (dng uncompressed) the purple line becomes black with bad pixel turned off i have yet to try it with it on

[gac123]

example of the highlights (which look ok!) on the body of the guitar

800/3200

[gac123]

oh, and no black frames! :)

[gac123]

but still, all raw corrections besides bad pixel fix and diso on/off don't do anything at all in the exports..