darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.73k stars 1.14k forks source link

[lut3d gmic] error - saving cache LUT (does the cache folder exist?) #16350

Open justin-ashworth-research opened 8 months ago

justin-ashworth-research commented 8 months ago

Describe the bug

On macOS 14.3.1, darktable release v4.6, G'MIC (via homebrew) is enabled in the release, and can read/apply luts in gmic_cluts.gmz.

But lut3d chokes simply on caching the decompressed luts.

[lut3d gmic] error - saving cache LUT (does the cache folder exist?)

The intended full target path is not included in the error message (but it should be!) Normal cache dirs in ~/.cache/ and ~/Library/ are present, but not utilized. A relevant configurable setting for darktable cache locations is not apparent.

Perhaps macOS package maintainer(s) can add simple fixes? I am still working on a local build [some build deps choking atm]

Steps to reproduce

  1. Install gmic via homebrew
  2. Download gmic_cluts.gmz
  3. Open a build of Darktable that looks for gmic [e.g. 4.6.0]
  4. Set the LUTs folder in settings:processing
  5. Open the gmic cluts in lut3d

Expected behavior

darktable should succeed in caching decompressed LUTs, and/or explicitly report failed path locations (e.g. in CLI messages)

Logfile | Screenshot | Screencast

[lut3d gmic] error - saving cache LUT (does the cache folder exist?)

Commit

No response

Where did you obtain darktable from?

downloaded from www.darktable.org

darktable version

4.6

What OS are you using?

Mac

What is the version of your OS?

macOS 14.3.1

Describe your system?

No response

Are you using OpenCL GPU in darktable?

None

If yes, what is the GPU card and driver?

No response

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

No response

zisoft commented 8 months ago

Check your CMAKE output, I guess it reports that GMIC was not found.

Homebrew does not install a gmic.dylib, only gmic.a.

Please consider raising an upstream bug at the homebrew gmic maintainers.

zisoft commented 8 months ago

Related: #14670

justin-ashworth-research commented 8 months ago

The version 4.6.0 release for macOS was apparently built with G'MIC, and does properly interpret/apply the CLUTs from gmic_cluts.gmz. But something chokes on caching them. Hence I can use the CLUTs, but inefficiently, and with this vague error message.

I was able to build from source (4.7) including GMIC support, and installed all of the GMIC .so's directly from source as well so that the darktable build can find them. It doesn't seem to fix the problem

But my quick build is unusably slow to go much further (perhaps some resource/OpenCL issue)

I noticed the CLI exception code in src/iop/lut3dgmic.cpp (which produces the stderr) doesn't actually check the cache path (misleading?)

zisoft commented 8 months ago

So, I made some investigations on my Mac by setting a breakpoint at line 73:

https://github.com/darktable-org/darktable/blob/018de4a124e88a7ce2cc07ce821160bae6fc1c20/src/iop/lut3dgmic.cpp#L73C1-L81C4

The cache directory on a Mac is /Users/<username>/.cache/gmic and it is automatically created at darktable start, good. When it comes to create the file in the cache, the exception is thrown and no file is created.

It looks like the Mac doesn't like the ,uchar at the end of the command. When I remove that, the file in the cache directory is created successfully. But this has for sure other side effects.

Further investigation is needed...

justin-ashworth-research commented 8 months ago

That does seem to work! The caching proceeds with that fix, and gives a minor speedup for applying CLUTs from gmic_cluts.gmz. But still glacially slow, compared to application of HALDCluts (.pngs on disk), which is nearly instantaneous.

Perhaps indeed something is awry in how GMIC module is working? Is the math really that heavy?

dev-dev-dev-dev-dev-dev-dev-dev commented 8 months ago

For what it's worth, I'm running into this same issue on arch linux, both 4.6.1 and current git master.

zisoft commented 8 months ago

@dev-dev-dev-dev-dev-dev-dev-dev thank you, so not a macOS specific issue.

github-actions[bot] commented 6 months ago

This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.