search

darktable-org / darktable

darktable is an open source photography workflow application and raw developer
https://www.darktable.org
GNU General Public License v3.0
9.54k stars 1.13k forks source link

DT 3.3 git master on Windows: LUT3D module disappeared #6936

Closed Mark-64 closed 3 years ago

Mark-64 commented 3 years ago

Last DT 3.3 git master, build on Windows 10 on a fully updated MSYS2.

LUT3D module is no longer there, not evean searching for it immagine

However in the build log, lut3d.c and lut3dgmic.cpp seem to be compiled normally.

Any clue ?

EDIT: On Ubuntu LUT3D is present

TurboGit commented 3 years ago

The build on windows is probably missing the libgmic.

chhil commented 3 years ago

I am on the latest build and latest msys, lut3d is available.

Maybe related to

https://github.com/darktable-org/darktable/blob/master/packaging/windows/BUILD.txt#L19

    Install optional libraries and dependencies:
  $ pacman -S mingw-w64-x86_64-gmic
Mark-64 commented 3 years ago

Nope immagine

Also, cmake finds gmic and as I said the source files are compiled (they would not be if gmic is not found)

chhil commented 3 years ago

Gmic package is optional, so it would still compile, I wondered if it broke, but doesn't seem to be the case.

TurboGit commented 3 years ago

So nothing wrong just a missing optional dependency.

Mark-64 commented 3 years ago

@TurboGit sorry but for me the issue is not solved. Gmic is there, I even tried to reistall it. Cmake finds it. I built DT on the same system hundreds times amd lut3d was alway there.

I will try to bisect the commits. Can you please reopen the issue ?

TurboGit commented 3 years ago

Maybe there is another dependency... Don't remember... @phweyland : can you remind us the needed dependency to have the Lut module ?

phweyland commented 3 years ago

If I remember well lut3d module fails to initialize when the run time gmic library is not found. In case of windows, this library should be included in the dt package... I've still my former windows alive. I'll try to make a build there again.

phweyland commented 3 years ago

Doesn't start well:

$ pacman -S mingw-w64-x86_64-gmic
error: target not found: mingw-w64-x86_64-gmic
chhil commented 3 years ago

image

phweyland commented 3 years ago

Yes, my msys is very old, needs to update, key issues,... normal life...

phweyland commented 3 years ago

No luck so far. Some warnings and errors I don't master...

BTW, if you don't use gmic, you may want to exclude it in DefineOptions.cmake. That will compile lut3d without GMIC and may help waiting for a solution. EDIT : option(USE_GMIC "Use G'MIC image processing framework." OFF)

EDIT2 : restarting from scratch...

phweyland commented 3 years ago

For the time being:

Building msys2 and dt from scratch gives me the same issue. liblut3d.dll, libgmic.dll, lut3d.cl are present in dt folder.

The crash is here: [iop_load_module] failed to open operationlut3d': 'C:\msys64\opt\darktable\lib\darktable/plugins\liblut3d.dll': The specified module could not be found.`

The message may be misleading. I had the same message when the libgmic.dll was missing.

phweyland commented 3 years ago

On msys2 the gmic version is 2.9. Seems to be broken for dt. So far I haven't found the way to downgrade to 2.8 or 2.7 from msys2. If somebody knows ...

phweyland commented 3 years ago

I'm stuck here:

[iop_load_module] loading ioplut3d' from C:\msys64\opt\darktable\lib\darktable/plugins\liblut3d.dll`

module->module = g_module_open(libname, G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL);

The previous call fails,

[iop_load_module] failed to open operationlut3d': 'C:\msys64\opt\darktable\lib\darktable/plugins\liblut3d.dll': The specified module could not be found.`

liblut3d.dll is there. libgmic.dll is in the darktable folder as usual.

I have no idea to debug this, but if someone has any advice, I can try to follow...

@dtschump, gmic 9.x seems not to work anymore under windows, while we don't get any error during the build. The module associated with libgmic.dll doesn't load. Any idea or person to help on this ?

parafin commented 3 years ago

I suggest to try a tool like https://dependencywalker.com/ (something like ldd or lddtree on Linux) and check out what it shows for liblut3d.dll. Probably some missing dependency, which is detected only at runtime, and not needed for linking.

phweyland commented 3 years ago

Debian has a very old version of gmic, but I didn't notice that it's even not possible to install any of the more recent GMIC versions on Debian Bullseye and Sid. I have libilmbase24 and libilmbase25 available but GMIC requests for libilmbase23(>=2.2.1)... So I'm not able to check if lut3d still works with GMIC on debian (the min version of GMIC for dt is 2.7).

@parafin. Thanks for the tip. I'll look at it and make a try.

phweyland commented 3 years ago

"depends" gives me this (related to gmic) :

image

and

 pacman -S mingw-w64-x86_64-opencv
warning: mingw-w64-x86_64-opencv-4.5.0-1 is up to date -- reinstalling

and under darktable EDIT (executable) folder I have no file responding to *opencv*

Should be including somewhere in dt build ??

chhil commented 3 years ago

image

phweyland commented 3 years ago

And under /opt/darktable ? You may have kept the library from a previous build ?

chhil commented 3 years ago

I build my build environment less than a month ago. Haven't had issues with building or lut module.

To search for already installed packages: 

$ pacman -Qs opencv
local/mingw-w64-x86_64-opencv 4.4.0-1
    Open Source Computer Vision Library (mingw-w64)

Some useful commands on this page https://wiki.archlinux.org/index.php/pacman#Querying_package_databases

Possibly it needs 4.4.0 you have 4.5.0 installed.

dtschump commented 3 years ago

Maybe some hints:

phweyland commented 3 years ago

@dtschump , thanks for jumping in.

Msys2 is providing 2.9.0-3. So we have to wait for the version 2.9.3...

Meanwhile I'm trying to find the way to include opencv in the dt package et check if that's the unique issue. So far the procedure to get fftw3 library doesn't work for opencv...

phweyland commented 3 years ago

This sequence works:

    if(WIN32)
      if(GMIC_LIBRARY)
        find_library(FFTW_LIBRARY NAMES fftw3)
        if(NOT FFTW_LIBRARY)
          message(STATUS "missing GMIC dependency fftw3")
        else()
          message(STATUS "found GMIC dependencies ${FFTW_LIBRARY}")
          list(APPEND GMIC_LIBRARY ${FFTW_LIBRARY})
        endif(NOT FFTW_LIBRARY)
        find_library(OPENCV_CORE_LIBRARY NAMES libopencv_core)
        find_library(OPENCV_VIDEOIO_LIBRARY NAMES libopencv_videoio)
        if(NOT OPENCV_CORE_LIBRARY OR NOT OPENCV_VIDEOIO_LIBRARY)
          message(STATUS "missing GMIC dependencies OpenCV")
        else()
          message(STATUS "GMIC dependencies found: ${OPENCV_CORE_LIBRARY} and ${OPENCV_VIDEOIO_LIBRARY}")
          list(APPEND GMIC_LIBRARY ${OPENCV_CORE_LIBRARY})
          list(APPEND GMIC_LIBRARY ${OPENCV_VIDEOIO_LIBRARY})
        endif(NOT OPENCV_CORE_LIBRARY OR NOT OPENCV_VIDEOIO_LIBRARY)
      ENDIF(GMIC_LIBRARY)
    endif(WIN32)

I can find the opencv libraries in the different build.make along with libfftw3.

But the libraries are not copied into darktable/bin. I miss something ... I need help ... Thanks in advance

phweyland commented 3 years ago

Found it. opencv are there: image

But as @chhil commented the version is not recognized by GMIC. Changing manually the name (450 to 440) makes lut3d appear again (and, better, it works):

image

@TurboGit I can issue a PR for that, letting the manual change in user's hand. Of course when GMIC version will change, this PR will have to be reverted. Do you want me to issue it even so ?

chhil commented 3 years ago

Great. Good job on sorting it out.

chhil commented 3 years ago

The dependency check that you performed on libgmic.dll was it with the latest gmic from msys or and older version ?

If the latest one is referencing the 440 version then there is a problem with the msys packages.

phweyland commented 3 years ago

The dependency check that you performed on libgmic.dll was it with the latest gmic from msys or and older version ?

If the latest one is referencing the 440 version then there is a problem with the msys packages.

The latest (and as far as I know, the unique) GMIC version available on msys2. Msys2 provides dependencies as defined by the GMIC packager. If I'm not mistaken, the issue comes from there, and should be solved when GMIC is updated to 9.3 in msys2.

phweyland commented 3 years ago

Note: some times ago it was possible on msys2 to retrieve older version of package. The maintainer(s) changed, the look of the site too. And the former versions seem to have disappear. :(

phweyland commented 3 years ago

And the former versions seem to have disappear.

Not true ! There is no handy link but you can find all packages' versions here. :)

If one specifies a valid version pacman is maybe able to retrieve it (I haven't tested). At least one can download the required version and install it locally.

MStraeten commented 3 years ago

maybe the old description dealing with lensfun can be used as a template:

NOTE: at the moment, the lensfun version provided by MSYS2 (0.3.95-1) is broken. If you get errors in the build process, manually downgrade lensfun to version 0.3.2-4:
    Download the file mingw-w64-x86_64-lensfun-0.3.2-4-any.pkg.tar.xz from http://repo.msys2.org/mingw/x86_64/
    Put the file in your MSYS2 home folder, then from MSYS2 terminal:
    $ pacman -U mingw-w64-x86_64-lensfun-0.3.2-4-any.pkg.tar.xz
    Also in order to prevent pacman from updating the lensfun package, open the file /etc/pacman.conf, find the line:
    #Pacman won’t upgrade packages listed in IgnorePkg and members of IgnoreGroup
    and below, add the following line
    IgnorePkg = mingw-w64-x86_64-lensfun
    Copy the lensfun folder and the lensfun-0.3.2-py3.6.egg-info file from /mingw64/lib/python3.6/site-packages to /mingw64/lib/python3.8/site-packages.
TurboGit commented 3 years ago

Do you want me to issue it even so ?

This is fully Windows specific right? If so I think it would be good to have this for the 3.4 release to ensure Windows users will have Lut3D activated and working. If the change is only about the CMakeList you posted above it seems correct to me. TIA.

phweyland commented 3 years ago

I think it would be good to have this for the 3.4 release

I'll do.

BTW I've built gmic 3.9 and installed libgmic. lut3d works with gmic features.

johnny-bit commented 3 years ago

Fixed in #6976 (PR wasn't connected) - closing