LUT Maker: file format selector doesn't show up on MacOS

[elizarification]

Hey Nate! Thanks for fixing the mac export bug. Works flawlessly now. I've been trying to use my generated LUTs with Resolve but it seems to recognise only .cube format. Is it possible to add different formats to the export functionality of LUT Maker? Am I missing something? Any advice on how to convert them?

[cessen]

Hi @elizarification!

Yes, I'm more than happy to add additional LUT formats. Currently both .spi1d and .cube are supported (in the export dialog, you can select which format to export to in the dropdown).

However, there are (annoyingly) two almost identical but slightly different LUT formats that both use the .cube extension. Of those two, LUT Maker currently only supports the IRIDAS cube format, and I don't know if Resolve also supports that or not.

The other cube format is Resolve's own custom cube format, which I'm sure they do support, of course. Unfortunately, Blackmagic Design has not (as far as I know) published any official specs for the format. But it's pretty simple and there is this post on their forums. So I should be able to get something working.

I'll open a new issue to collect LUT formats that people want support for.

[cessen]

Created issue #8 to track this.

[elizarification]

Hi!

Regarding the export format. I don't see an option you mention. There's no choice in the system's file manager window, and I don't see in the LUT-maker itself (there's only file menu with quit option). I'll attach the screenshots.

But I tried just manually typing in the .cube extension and it kinda worked. At least there were no errors and later there were no errors in Resolve. It has an effect on the footage and the second LUT undoes this effect but I'm not sure it's working properly at all. It just gets everything darker.

I wonder if the estimate is correct. If look at the screenshot it says 'average error: NaN'.

I'll experiment some more later and try using LUTs in Blender.

[cessen]

This looks like two separate bugs:

1. The file format selector isn't showing up on Mac OS.
2. The estimated curve is definitely not right. The error should never be NaN.

To help me solve both of these, would it be possible for you to share:

1. The .cube files you've created by changing the extension? I'm curious if it's actually exporting to .cube or not.
2. The image files you're using that result in the NaN error.

[cessen]

Investigating the first bug, it looks like the file dialog library I'm using doesn't support file-type drop downs in the save dialog on MacOS. Additionally, even on the other platforms, it doesn't provide enough information back to my code to distinguish two formats with the same extension. So I may need to rethink things there a bit. Probably I'll add a drop down in the main UI next to the export buttons for selecting the export format.

I could still definitely use the image files that cause the NaN though! Would love to track that down and get it fixed. So if you get the chance, and are able to share them, I would really appreciate it!

[elizarification]

No problem.

I'll attach the files.

There's another thing that may be relevant. 0.2.2 version (the one that crashes during export) estimates a different curve from the same images. But probably wrong, too. In this case the error is Inf.

The files:

cubes.zip

The link to the image sets: https://mega.nz/file/bQd0CbIS#MMCpLenTGsAe4bUvpy0htkOBHT0FXWNAXMXraA9m-4w

[cessen]

Thanks so much! This is really a huge help.

There's another thing that may be relevant. 0.2.2 version (the one that crashes during export) estimates a different curve from the same images.

Yeah, I made some improvements in the estimator between those versions, so getting a different curve is expected. Indeed, Inf is also something that shouldn't ever happen.

(I have additional potential improvements to the estimator that I want to test out, but at the moment I'm focusing my time on the next video in the color series. I'm a lot slower at making videos than making software apparently, ha ha.)

[elizarification]

I loved your first video!

Look, maybe you should teach me how to start LUT-maker in a debug mode (if there's one)? So it would output all it's doing to the terminal? I'm in.

Also, Ian's tutorial said that LUT-maker relies on shutter speed to estimate the bracketed exposure. But, if it's not entirely true, you should know that I use (and own) exclusively vintage lenses. So in the metadata the aperture is always 0. Maybe this is what messes up the calculation. I took all the photos at 5.6 but it's not reflected in the metadata.

[cessen]

Thanks for the files! I'll look into this soon.

So in the metadata the aperture is always 0.

Oh! That could definitely be it. It actually accounts for ISO, aperture, and shutter speed together. In practice, you should only be changing the shutter speed, because ISO and aperture aren't invariant in the ways you want for this kind of estimation. But it does account for all of them in the software, so I can imagine an aperture of zero may be resulting in a divide by zero somewhere, which would definitely result in a NaN.

If that's the case, this should be an easy fix (i.e. if the aperture is zero, just ignore it).

Thanks for the tip!

[elizarification]

Great, looking forward to it!

Of course, I didn't change the ISO or aperture.

If that's the case, this should be an easy fix (i.e. if the aperture is zero, just ignore it).

Or you could add a menu to set the aperture manually, if it helps with the estimate. If it does not, then of course to hell with it. Anyway, I'm happy to help.

Thanks a lot! I really appreciate your effort and attention.

[cessen]

I believe I have a fix for the NaN issue. Here's a dev build with the fix for you to test:

https://github.com/EatTheFuture/image_tools/releases/tag/dev_build-2022_02_16-3c19e1d

A couple of notes:

1. Your .cube files are, indeed, proper .cube files. I'm curious to know if they work in Resolve or not, so if you get the chance, please let me know!
2. Your bracketed exposures wobble around a lot from shot to shot, which is likely to result in a very poor estimate of the transfer function. You really need to use a tripod, or some other way of keeping the camera in a fixed position. It doesn't have to be perfect, but if the ratio of the frame taken up by e.g. bright sky vs everything else is changing from shot to shot, that's going to throw off the estimate quite a bit.

Or you could add a menu to set the aperture manually, if it helps with the estimate. If it does not, then of course to hell with it.

As long as the aperture is the same between exposures, then it doesn't matter. Having said that, I do eventually want to add an option for manually specifying the relative exposures of the images, for cases where metadata isn't included in the files at all. But in this case, you're fine.

Forgot to respond to this:

but I'm not sure it's working properly at all. It just gets everything darker.

Generally speaking, the estimated "to linear" LUTs are going to darken colors, and the estimated "from linear" LUTs will brighten them. It's a side effect of decoding/encoding the colors to/from linear color. So that's expected.

[elizarification]

I tried the new build. Everything works!

1. The LUTs work in Resolve.
2. Somehow it was not obvious to me that fixing the camera is important. Of course it wobbles -- it's a 50mm lens times 1.4 crop factor, so about 70mm handheld. I made a new set for the new estimate shooting from a tripod.
3. The error is quantified now:
4. Understood regarding the darkening effect. It is present now but everything looks much better.

But I have a suggestion. Maybe we should start a thread for the users' questions. Because I watched Ian's tutorial three times, your first video two times, read the whole chapter on color management, and still I'm on shaky grounds. Not to mention the tripod thing. I have questions, as other users would do, too. So maybe it's a good idea to collect them in one place. Hopefully, knowing common questions and misconceptions would help you to better design your explainer videos, too.

Or should I just ask them here?

Anyway, thank you.

[cessen]

Great to hear! I'm still going to leave this issue open until I implement a proper way to select the desired export format. But good to know that everything is working.

But I have a suggestion. Maybe we should start a thread for the users' questions. Because I watched Ian's tutorial three times, your first video two times, read the whole chapter on color management, and still I'm on shaky grounds.

Ah! That's a great idea. Prompted by you, I've now enabled the GitHub discussions feature for this repository. Questions are more than welcome there, although at the moment I can't promise timely responses.

More generally, though, I do want to make in-depth videos about how to use these tools. (Indeed, the documentation right now is essentially nonexistent, and Ian's video was more like a brief "getting started" video than a proper tutorial.) I've been putting that off until I finish the next video in the color series, because those first two videos combined will basically cover all the background needed to understand the current capabilities of the tools. But hopefully soon!

[cessen]

Added an explicit export format dropdown in the main UI in df605426c90d26181dcf0d4e6f1e81043151b16b.

A dev build is available for testing: https://github.com/EatTheFuture/image_tools/releases/tag/dev_build-2022_05_24-f2419db