darktable-org / rawspeed

fast raw decoding library
GNU Lesser General Public License v2.1
353 stars 113 forks source link

Compiling in Visual Studio? #261

Open TomArrow opened 3 years ago

TomArrow commented 3 years ago

I'm trying to use this in a project I have in Visual Studio, but as a C++ beginner I'm at the end of my wits for now.

The header files I managed to make work by doing this in a newly created rawspeedconfig.h, based on some info from various places:

#ifdef _MSC_VER
#define __attribute__(x) 
#define __builtin_unreachable(x)
#define ASAN_REGION_IS_POISONED(x) 0
#define __PRETTY_FUNCTION__ __FUNCSIG__
#endif

I also copied a bunch of the code from config.h.in to my rawspeedconfig.h. After that the only remaining issue after compiling was unresolved symbols (all the actual function bodies etc).

But when I include all the other sources, it will just spit out a billion syntax errors and die. Most if it seems to be due to some gcc related stuff. I wanted to use this code as a submodule in git so I don't want to just go rogue and replace all the GCC stuff (and I'm not sure if I should either, since I would likely end up breaking something).

Is there any way at all to compile this code without gcc?

I was thinking of compiling with gcc and using the static library inside Visual Studio but I then read that this is impossible, so I'm back to zero.

LebedevRI commented 3 years ago

Only LLVM/Clang and GCC compilers are supported. Microsoft's compiler is not supported, and i'm not sure what hacks/changes would be needed to make it work.

I would recommend trying to use clang-cl, see https://clang.llvm.org/docs/MSVCCompatibility.html

TomArrow commented 3 years ago

Thanks for your answer.

I tried opening the folder in Visual Studio (2019 btw) as a CMake project, set the config to clang and I got this error: STUDIO/2019/COMMUNITY/VC/Tools/Llvm/bin/clang-cl.exe has no C++14 support.

Do I need to take a more manual approach, so to speak?

I also got some error about a config.json not being able to be opened initially.

Then later I tried it again by just creating a static library project and adding the files. That made the C++14 error go away, but instead I got an error about that processor cores function in some CMakeLists.txt. Can you tell me how and where I must implement this? Is this some kind of "Cmake function" or whatever? Sorry if that sounds dumb. Or do I just create a normal C++ function like that?

LebedevRI commented 3 years ago

Honestly, i have no idea. I have never used Visual Studio, and i have never used clang-cl. I can guess that even with clang-cl, as per https://clang.llvm.org/docs/UsersManual.html#id9, dash-style flags aren't avaliable, and forward-slash-style flags must be used. I.e. https://github.com/darktable-org/rawspeed/blob/62f607357648ebce956f57b31b37e688afc86d0d/cmake/compiler-flags.cmake#L12 should be checking for /std:14, when the compiler is clang-cl.

TomArrow commented 3 years ago

Going back to the original idea of compiling with VS normally (since clang doesn't seem all that well implemented in VS, not giving meaningful errors for example), would you consider doing a change or accepting a pull request to the code where the GCC specific stuff is turned into a macro that is defined conditionally based on the compiler? So for example, in the code itself, __attribute__((const)) would become ATTRIBUTE_CONST and then in some config file or something, it would be like this:

#define ATTRIBUTE_CONST __attribute__((const))

and then I could just do my own define in Visual studio that goes:

#define ATTRIBUTE_CONST 

which would in effect just turn it into empty space.

Because the problem I have is that the ones with double brackets like __attribute__((const)) can't get replaced via macro, since the double bracket is interpreted by Visual Studio as a syntax error.

Of course I could just do my own fork and leave it like that there, but then I that would create long-term problems with maintaining the change etc., whereas implementing this directly into the main codebase would provide a tiny step towards being more generally compatible without sacrificing the original functionality this stuff was meant to provide. At least that's the way I see it from my limited experience. Let me know what you think.

LebedevRI commented 3 years ago

I would be open/sympathetic to supporting clang-cl. I'm not sure i'm interested in supporting VS compiler.

TomArrow commented 3 years ago

Well, it wouldn't necessarily mean you have to support it explicitly, it would just get rid of some major hurdles that make it impossible. But alright, I'll respect your decision.

I'll try and look into the clang thing if I find the time and motivation. Thanks.

TomArrow commented 3 years ago

So when I try to properly use the CMAKE gui and I specify the platform toolset as "ClangCL", I get a bit further than usual, but I end up with the same problem with the compiler flag test. Here's the exact errors I get:

CMake Warning at cmake/build-type.cmake:3 (message):
  CMAKE_BUILD_TYPE is not defined!
Call Stack (most recent call first):
  CMakeLists.txt:185 (include)

WARNING: Defaulting to CMAKE_BUILD_TYPE=Sanitize. Use ccmake to set a proper value.

This error doesn't stop the build, it's just marked in red, so I thought I'd mention it.

Here's the fatal one:

CMake Error at cmake/compiler-flags.cmake:14 (message):
  The compiler C:/Program Files (x86)/Microsoft Visual
  Studio/2019/Community/VC/Tools/Llvm/x64/bin/clang-cl.exe has no C++14
  support.  Please use a different C++ compiler.
Call Stack (most recent call first):
  CMakeLists.txt:208 (include)

I tried to comment out this line:

message(FATAL_ERROR "The compiler ${CMAKE_CXX_COMPILER} has no C++14 support. Please use a different C++ compiler.")

and it got a bit further but ended up failing at:

CMake Error at cmake/Modules/GoogleTest.cmake:12 (message):
  CMake step for googletest failed: 1
Call Stack (most recent call first):
  cmake/src-dependencies.cmake:8 (include)
  CMakeLists.txt:232 (include)

I strongly suspect I shouldn't be editing that line out, but at least it helped me find another error.

TomArrow commented 3 years ago

Not sure if it helps in any way but here's the -help output of the clang-cl.exe:

OVERVIEW: clang LLVM compiler

USAGE: clang-cl.exe [options] file...

CL.EXE COMPATIBILITY OPTIONS:
  /?                      Display available options
  /arch:<value>           Set architecture for code generation
  /Brepro-                Write current time into COFF output (default)
  /Brepro                 Do not write current time into COFF output (breaks link.exe /incremental)
  /clang:<arg>            Pass <arg> to the clang driver
  /C                      Do not discard comments when preprocessing
  /c                      Compile only
  /d1PP                   Retain macro definitions in /E mode
  /d1reportAllClassLayout Dump record layout information
  /diagnostics:caret      Enable caret and column diagnostics (default)
  /diagnostics:classic    Disable column and caret diagnostics
  /diagnostics:column     Disable caret diagnostics but keep column info
  /D <macro[=value]>      Define macro
  /EH<value>              Set exception handling model
  /EP                     Disable linemarker output and preprocess to stdout
  /execution-charset:<value>
                          Set runtime encoding, supports only UTF-8
  /E                      Preprocess to stdout
  /fallback               Fall back to cl.exe if clang-cl fails to compile
  /FA                     Output assembly code file during compilation
  /Fa<file or dir/>       Set assembly output file name (with /FA)
  /Fe<file or dir/>       Set output executable file name
  /FI <value>             Include file before parsing
  /Fi<file>               Set preprocess output file name (with /P)
  /Fo<file or dir/>       Set output object file (with /c)
  /fp:except-             
  /fp:except              
  /fp:fast                
  /fp:precise             
  /fp:strict              
  /Fp<file>               Set pch file name (with /Yc and /Yu)
  /GA                     Assume thread-local variables are defined in the executable
  /Gd                     Set __cdecl as a default calling convention
  /GF-                    Disable string pooling
  /GF                     Enable string pooling (default)
  /GR-                    Do not emit RTTI data
  /Gregcall               Set __regcall as a default calling convention
  /GR                     Emit RTTI data (default)
  /Gr                     Set __fastcall as a default calling convention
  /GS-                    Disable buffer security check
  /GS                     Enable buffer security check (default)
  /Gs                     Use stack probes (default)
  /Gs<value>              Set stack probe size (default 4096)
  /guard:<value>          Enable Control Flow Guard with /guard:cf, or only the table with /guard:cf,nochecks
  /Gv                     Set __vectorcall as a default calling convention
  /Gw-                    Do not put each data item in its own section (default)
  /Gw                     Put each data item in its own section
  /GX-                    Deprecated (like not passing /EH)
  /GX                     Deprecated; use /EHsc
  /Gy-                    Do not put each function in its own section (default)
  /Gy                     Put each function in its own section
  /Gz                     Set __stdcall as a default calling convention
  /help                   Display available options
  /imsvc <dir>            Add <dir> to system include search path, as if in %INCLUDE%
  /I <dir>                Add directory to include search path
  /J                      Make char type unsigned
  /LDd                    Create debug DLL
  /LD                     Create DLL
  /link <options>         Forward options to the linker
  /MDd                    Use DLL debug run-time
  /MD                     Use DLL run-time
  /MTd                    Use static debug run-time
  /MT                     Use static run-time
  /O1                     Optimize for size  (like /Og     /Os /Oy /Ob2 /GF /Gy)
  /O2                     Optimize for speed (like /Og /Oi /Ot /Oy /Ob2 /GF /Gy)
  /Ob0                    Disable function inlining
  /Ob1                    Only inline functions explicitly or implicitly marked inline
  /Ob2                    Inline functions as deemed beneficial by the compiler
  /Od                     Disable optimization
  /Og                     No effect
  /Oi-                    Disable use of builtin functions
  /Oi                     Enable use of builtin functions
  /openmp-                Disable OpenMP support
  /openmp:experimental    Enable OpenMP support with experimental SIMD support
  /openmp                 Enable OpenMP support
  /Os                     Optimize for size
  /Ot                     Optimize for speed
  /Ox                     Deprecated (like /Og /Oi /Ot /Oy /Ob2); use /O2
  /Oy-                    Disable frame pointer omission (x86 only, default)
  /Oy                     Enable frame pointer omission (x86 only)
  /O<flags>               Set multiple /O flags at once; e.g. '/O2y-' for '/O2 /Oy-'
  /o <file or dir/>       Deprecated (set output file name); use /Fe or /Fe
  /P                      Preprocess to file
  /Qvec-                  Disable the loop vectorization passes
  /Qvec                   Enable the loop vectorization passes
  /showFilenames-         Do not print the name of each compiled file (default)
  /showFilenames          Print the name of each compiled file
  /showIncludes           Print info about included files to stderr
  /source-charset:<value> Set source encoding, supports only UTF-8
  /std:<value>            Set C++ version (c++14,c++17,c++latest)
  /TC                     Treat all source files as C
  /Tc <file>              Treat <file> as C source file
  /TP                     Treat all source files as C++
  /Tp <file>              Treat <file> as C++ source file
  /utf-8                  Set source and runtime encoding to UTF-8 (default)
  /U <macro>              Undefine macro
  /vd<value>              Control vtordisp placement
  /vmb                    Use a best-case representation method for member pointers
  /vmg                    Use a most-general representation for member pointers
  /vmm                    Set the default most-general representation to multiple inheritance
  /vms                    Set the default most-general representation to single inheritance
  /vmv                    Set the default most-general representation to virtual inheritance
  /volatile:iso           Volatile loads and stores have standard semantics
  /volatile:ms            Volatile loads and stores have acquire and release semantics
  /W0                     Disable all warnings
  /W1                     Enable -Wall
  /W2                     Enable -Wall
  /W3                     Enable -Wall
  /W4                     Enable -Wall and -Wextra
  /Wall                   Enable -Weverything
  /WX-                    Do not treat warnings as errors (default)
  /WX                     Treat warnings as errors
  /w                      Disable all warnings
  /X                      Do not add %INCLUDE% to include search path
  /Y-                     Disable precompiled headers, overrides /Yc and /Yu
  /Yc<filename>           Generate a pch file for all code up to and including <filename>
  /Yu<filename>           Load a pch file and use it instead of all code up to and including <filename>
  /Z7                     Enable CodeView debug information in object files
  /Zc:alignedNew-         Disable C++17 aligned allocation functions
  /Zc:alignedNew          Enable C++17 aligned allocation functions
  /Zc:char8_t-            Disable char8_t from c++2a
  /Zc:char8_t             Enable char8_t from C++2a
  /Zc:dllexportInlines-   Do not dllexport/dllimport inline member functions of dllexport/import classes
  /Zc:dllexportInlines    dllexport/dllimport inline member functions of dllexport/import classes (default)
  /Zc:sizedDealloc-       Disable C++14 sized global deallocation functions
  /Zc:sizedDealloc        Enable C++14 sized global deallocation functions
  /Zc:strictStrings       Treat string literals as const
  /Zc:threadSafeInit-     Disable thread-safe initialization of static variables
  /Zc:threadSafeInit      Enable thread-safe initialization of static variables
  /Zc:trigraphs-          Disable trigraphs (default)
  /Zc:trigraphs           Enable trigraphs
  /Zc:twoPhase-           Disable two-phase name lookup in templates (default)
  /Zc:twoPhase            Enable two-phase name lookup in templates
  /Zd                     Emit debug line number tables only
  /Zi                     Like /Z7
  /Zl                     Do not let object file auto-link default libraries
  /Zp                     Set default maximum struct packing alignment to 1
  /Zp<value>              Set default maximum struct packing alignment
  /Zs                     Syntax-check only

OPTIONS:
  -###                    Print (but do not run) the commands to run for this compilation
  --analyze               Run the static analyzer
  -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang
                          Trivial automatic variable initialization to zero is only here for benchmarks, it'll eventually be removed, and I'm OK with that because I'm only using it to benchmark
  -faddrsig               Emit an address-significance table
  -fansi-escape-codes     Use ANSI escape codes for diagnostics
  -fblocks                Enable the 'blocks' language feature
  -fcf-protection=<value> Instrument control-flow architecture protection. Options: return, branch, full, none.
  -fcf-protection         Enable cf-protection in 'full' mode
  -fcolor-diagnostics     Use colors in diagnostics
  -fcomplete-member-pointers
                          Require member pointer base types to be complete if they would be significant under the Microsoft ABI
  -fcoverage-mapping      Generate coverage mapping to enable code coverage analysis
  -fcs-profile-generate=<directory>
                          Generate instrumented code to collect context sensitive execution counts into <directory>/default.profraw (overridden by LLVM_PROFILE_FILE env var)
  -fcs-profile-generate   Generate instrumented code to collect context sensitive execution counts into default.profraw (overridden by LLVM_PROFILE_FILE env var)
  -fdebug-compilation-dir <value>
                          The compilation directory to embed in the debug info.
  -fdebug-macro           Emit macro debug information
  -fdelayed-template-parsing
                          Parse templated function definitions at the end of the translation unit
  -fdiagnostics-absolute-paths
                          Print absolute paths in diagnostics
  -fdiagnostics-parseable-fixits
                          Print fix-its in machine parseable form
  -fgnuc-version=<value>  Sets various macros to claim compatibility with the given GCC version (default is 4.2.1)
  -fintegrated-cc1        Run cc1 in-process
  -flto=<value>           Set LTO mode to either 'full' or 'thin'
  -flto                   Enable LTO in 'full' mode
  -fmerge-all-constants   Allow merging of constants
  -fms-compatibility-version=<value>
                          Dot-separated value representing the Microsoft compiler version number to report in _MSC_VER (0 = don't define it (default))
  -fms-compatibility      Enable full Microsoft Visual C++ compatibility
  -fms-extensions         Accept some non-standard constructs supported by the Microsoft compiler
  -fmsc-version=<value>   Microsoft compiler version number to report in _MSC_VER (0 = don't define it (default))
  -fno-addrsig            Don't emit an address-significance table
  -fno-builtin-<value>    Disable implicit builtin knowledge of a specific function
  -fno-builtin            Disable implicit builtin knowledge of functions
  -fno-complete-member-pointers
                          Do not require member pointer base types to be complete if they would be significant under the Microsoft ABI
  -fno-coverage-mapping   Disable code coverage analysis
  -fno-crash-diagnostics  Disable auto-generation of preprocessed source files and a script for reproduction during a clang crash
  -fno-debug-macro        Do not emit macro debug information
  -fno-delayed-template-parsing
                          Disable delayed template parsing
  -fno-integrated-cc1     Spawn a separate process for each cc1
  -fno-profile-generate   Disable generation of profile instrumentation.
  -fno-profile-instr-generate
                          Disable generation of profile instrumentation.
  -fno-profile-instr-use  Disable using instrumentation data for profile-guided optimization
  -fno-sanitize-address-poison-custom-array-cookie
                          Disable poisoning array cookies when using custom operator new[] in AddressSanitizer
  -fno-sanitize-address-use-after-scope
                          Disable use-after-scope detection in AddressSanitizer
  -fno-sanitize-address-use-odr-indicator
                          Disable ODR indicator globals
  -fno-sanitize-blacklist Don't use blacklist file for sanitizers
  -fno-sanitize-cfi-canonical-jump-tables
                          Do not make the jump table addresses canonical in the symbol table
  -fno-sanitize-cfi-cross-dso
                          Disable control flow integrity (CFI) checks for cross-DSO calls.
  -fno-sanitize-coverage=<value>
                          Disable specified features of coverage instrumentation for Sanitizers
  -fno-sanitize-memory-track-origins
                          Disable origins tracking in MemorySanitizer
  -fno-sanitize-memory-use-after-dtor
                          Disable use-after-destroy detection in MemorySanitizer
  -fno-sanitize-recover=<value>
                          Disable recovery for specified sanitizers
  -fno-sanitize-stats     Disable sanitizer statistics gathering.
  -fno-sanitize-thread-atomics
                          Disable atomic operations instrumentation in ThreadSanitizer
  -fno-sanitize-thread-func-entry-exit
                          Disable function entry/exit instrumentation in ThreadSanitizer
  -fno-sanitize-thread-memory-access
                          Disable memory access instrumentation in ThreadSanitizer
  -fno-sanitize-trap=<value>
                          Disable trapping for specified sanitizers
  -fno-standalone-debug   Limit debug information produced to reduce size of debug binary
  -fno-temp-file          Directly create compilation output files. This may lead to incorrect incremental builds if the compiler crashes
  -fobjc-runtime=<value>  Specify the target Objective-C runtime kind and version
  -forder-file-instrumentation
                          Generate instrumented code to collect order file into default.profraw file (overridden by '=' form of option or LLVM_PROFILE_FILE env var)
  -fprofile-exclude-files=<value>
                          Instrument only functions from files where names don't match all the regexes separated by a semi-colon
  -fprofile-filter-files=<value>
                          Instrument only functions from files where names match any regex separated by a semi-colon
  -fprofile-generate=<directory>
                          Generate instrumented code to collect execution counts into <directory>/default.profraw (overridden by LLVM_PROFILE_FILE env var)
  -fprofile-generate      Generate instrumented code to collect execution counts into default.profraw (overridden by LLVM_PROFILE_FILE env var)
  -fprofile-instr-generate=<file>
                          Generate instrumented code to collect execution counts into <file> (overridden by LLVM_PROFILE_FILE env var)
  -fprofile-instr-generate
                          Generate instrumented code to collect execution counts into default.profraw file (overridden by '=' form of option or LLVM_PROFILE_FILE env var)
  -fprofile-instr-use=<value>
                          Use instrumentation data for profile-guided optimization
  -fprofile-remapping-file=<file>
                          Use the remappings described in <file> to match the profile data against names in the program
  -fsanitize-address-field-padding=<value>
                          Level of field padding for AddressSanitizer
  -fsanitize-address-globals-dead-stripping
                          Enable linker dead stripping of globals in AddressSanitizer
  -fsanitize-address-poison-custom-array-cookie
                          Enable poisoning array cookies when using custom operator new[] in AddressSanitizer
  -fsanitize-address-use-after-scope
                          Enable use-after-scope detection in AddressSanitizer
  -fsanitize-address-use-odr-indicator
                          Enable ODR indicator globals to avoid false ODR violation reports in partially sanitized programs at the cost of an increase in binary size
  -fsanitize-blacklist=<value>
                          Path to blacklist file for sanitizers
  -fsanitize-cfi-canonical-jump-tables
                          Make the jump table addresses canonical in the symbol table
  -fsanitize-cfi-cross-dso
                          Enable control flow integrity (CFI) checks for cross-DSO calls.
  -fsanitize-cfi-icall-generalize-pointers
                          Generalize pointers in CFI indirect call type signature checks
  -fsanitize-coverage=<value>
                          Specify the type of coverage instrumentation for Sanitizers
  -fsanitize-hwaddress-abi=<value>
                          Select the HWAddressSanitizer ABI to target (interceptor or platform, default interceptor). This option is currently unused.
  -fsanitize-memory-track-origins=<value>
                          Enable origins tracking in MemorySanitizer
  -fsanitize-memory-track-origins
                          Enable origins tracking in MemorySanitizer
  -fsanitize-memory-use-after-dtor
                          Enable use-after-destroy detection in MemorySanitizer
  -fsanitize-recover=<value>
                          Enable recovery for specified sanitizers
  -fsanitize-stats        Enable sanitizer statistics gathering.
  -fsanitize-system-blacklist=<value>
                          Path to system blacklist file for sanitizers
  -fsanitize-thread-atomics
                          Enable atomic operations instrumentation in ThreadSanitizer (default)
  -fsanitize-thread-func-entry-exit
                          Enable function entry/exit instrumentation in ThreadSanitizer (default)
  -fsanitize-thread-memory-access
                          Enable memory access instrumentation in ThreadSanitizer (default)
  -fsanitize-trap=<value> Enable trapping for specified sanitizers
  -fsanitize-undefined-strip-path-components=<number>
                          Strip (or keep only, if negative) a given number of path components when emitting check metadata.
  -fsanitize=<check>      Turn on runtime checks for various forms of undefined or suspicious behavior. See user manual for available checks
  -fsplit-lto-unit        Enables splitting of the LTO unit.
  -fstandalone-debug      Emit full debug info for all types used by the program
  -fthin-link-bitcode=<value>
                          Write minimized bitcode to <file> for the ThinLTO thin link only
  -fthinlto-index=<value> Perform ThinLTO importing using provided function summary index
  -ftime-trace-granularity=<value>
                          Minimum time granularity (in microseconds) traced by time profiler
  -ftime-trace            Turn on time profiler. Generates JSON file based on output filename.
  -ftrivial-auto-var-init=<value>
                          Initialize trivial automatic stack variables: uninitialized (default) | pattern
  -fvirtual-function-elimination
                          Enables dead virtual function elimination optimization. Requires -flto=full
  -fwhole-program-vtables Enables whole-program vtable optimization. Requires -flto
  -gcodeview-ghash        Emit type record hashes in a .debug$H section
  -gcodeview              Generate CodeView debug information
  -gdwarf                 Generate source-level debug information with the default dwarf version
  -gline-directives-only  Emit debug line info directives only
  -gline-tables-only      Emit debug line number tables only
  -gno-inline-line-tables Don't emit inline line tables
  -miamcu                 Use Intel MCU ABI
  -mllvm <value>          Additional arguments to forward to LLVM's option processing
  -nobuiltininc           Disable builtin #include directories
  -print-supported-cpus   Print supported cpu models for the given target (if target is not specified, it will print the supported cpus for the default target)
  -Qunused-arguments      Don't emit warning for unused driver arguments
  -R<remark>              Enable the specified remark
  --target=<value>        Generate code for the given target
  --version               Print version information
  -v                      Show commands to run and use verbose output
  -W<warning>             Enable the specified warning
  -Xclang <arg>           Pass <arg> to the clang compiler
Ben-Voris commented 3 years ago

I know nothing about Microsoft C++ or clang, and only a very little about cmake, but I think cmake is trying to test if the compiler you've told it to use supports the C++14 standard. Presumably Darktable has code that uses C++14 features and has told cmake that C++14 is required. For an example, see How can I add a minimum compiler version requisite?.

These two pages might be helpful: Microsoft C++ language conformance table and this Enabling C++14 in clang in Visual Studio. If I understand the latter correctly and if I assume that it is accurate, the compiler should enable the latest support it provides. But, it might be worth getting the output of --version`` or to play different values for/std:`.

It could also be that clang-cl.exe does not report version information in a way that this cmake file understands.

LebedevRI commented 3 years ago

I tried to comment out this line:

You'll just need to add fallbacks to all such places in cmake to also try /windows-style flags as last-resort.

Not sure if it helps in any way but here's the -help output of the clang-cl.exe:

Not to me. Someone who cares about that platform and has direct access to that compiler should be doing such porting. I have already suggested how that particular place should be fixed:

  /std:<value>            Set C++ version (c++14,c++17,c++latest)
TomArrow commented 3 years ago

So, I've created my own fork and resorted, at least temporarily, to commenting out the part that sets the flag parameter. The reason is this:

# with just the CMAKE_CXX_STANDARD, try_compile() breaks:
#   https://gitlab.kitware.com/cmake/cmake/issues/16456
# with just the CMAKE_CXX_FLAGS, 'bundled' pugixml breaks tests
#   https://github.com/darktable-org/rawspeed/issues/112#issuecomment-321517003

The issue referenced when leaving out the CMAKE_CXX_FLAGS appears to be resolved on the CMake tracker so perhaps this is no longer necessary. But if there's evidence to the contrary, I'll be happy to give it another try.

In any case, what currently stops me from progressing is the googletest issue.

# Download and unpack googletest at configure time
set(GOOGLETEST_PREFIX "${RAWSPEED_BINARY_DIR}/src/external/googletest")
configure_file(${RAWSPEED_SOURCE_DIR}/cmake/Modules/GoogleTest.cmake.in ${GOOGLETEST_PREFIX}/CMakeLists.txt @ONLY)

execute_process(COMMAND ${CMAKE_COMMAND} -G "${CMAKE_GENERATOR}"
  -DALLOW_DOWNLOADING_GOOGLETEST=${ALLOW_DOWNLOADING_GOOGLETEST} -DGOOGLETEST_PATH:PATH=${GOOGLETEST_PATH} .
  RESULT_VARIABLE result
  WORKING_DIRECTORY ${GOOGLETEST_PREFIX}
)

if(result)
  message(FATAL_ERROR "CMake step for googletest failed: ${result}")
endif()

The error thrown by cmake is: CMake step for googletest failed: 1

I tried looking into the folder in the binary output folder src/external/googletest but found no source code or anything, just some Cmake stuff. There's also a log file and some other stuff in there but nothing that provides any clue to what went wrong.

Any ideas?

Specifically, there is a CMakeOutput.log in the CMakeFiles subdirectory of that folder that just says:

The system is: Windows - 6.1.7601 - AMD64
TomArrow commented 3 years ago

One thing I think might be the issue is commands like this:

set(GOOGLETEST_PATH "/usr/src/googletest" CACHE PATH
                    "Path to the googletest root tree. Should contain googletest and googlemock subdirs. And CMakeLists.txt in root, and in both of these subdirs")

set(PUGIXML_PATH "/usr/src/pugixml" CACHE PATH "Path to the pugixml root tree.")

set(ZLIB_PATH "/usr/src/zlib" CACHE PATH "Path to the zlib root tree.")

set(GOOGLEBENCHMARK_PATH "/usr/src/googlebenchmark" CACHE PATH
                    "Path to the googlebenchmark root tree.")

Obviously on Windows there can't be a /usr/src/...

My spontaneous solution to this would be to create a subfolder "ext" under "src" and add git submodules with the required libraries. Thoughts?

LebedevRI commented 3 years ago

So, I've created my own fork and resorted, at least temporarily, to commenting out the part that sets the flag parameter. The reason is this:

# with just the CMAKE_CXX_STANDARD, try_compile() breaks:
#   https://gitlab.kitware.com/cmake/cmake/issues/16456
# with just the CMAKE_CXX_FLAGS, 'bundled' pugixml breaks tests
#   https://github.com/darktable-org/rawspeed/issues/112#issuecomment-321517003

The issue referenced when leaving out the CMAKE_CXX_FLAGS appears to be resolved on the CMake tracker so perhaps this is no longer necessary. But if there's evidence to the contrary, I'll be happy to give it another try.

... which was part of cmake-3.8, while rawspeed requires 3.10. So yeah, i'd take a PR cleaning that up.

<...googletest...>

Works just fine in windows-based CI. As messages should have said, you'll need to either disable building tests, or point it at the location of sources, or allow downloading them. You probably want latter. I don't want to pull it via submodule.

TomArrow commented 3 years ago

Ah, interesting. Okay so my change would then basically be to remove this part:

CHECK_CXX_COMPILER_FLAG("-std=c++14" COMPILER_SUPPORTS_CXX14)
if(NOT COMPILER_SUPPORTS_CXX14)
  message(FATAL_ERROR "The compiler ${CMAKE_CXX_COMPILER} has no C++14 support. Please use a different C++ compiler.")
else()
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
  message(STATUS "Checking for -std=c++14 support - works")
endif()

And replace the message about needing both with a short note that that is no longer necessary. Does that make sense? I'm still very much beginning to understand how cmake works so wanna make sure I'm not breaking anything. So far I only commented that part out.

About googletest, thanks, I learned a new thing. Realized there's a checkbox I can set in the CMAKE GUI to activate the downloading and that worked. Curiously, it worked only for googletest and not for the other stuff like zlib, openmp, etc. When I put those checkboxes there, they just get ignored and after the next round of "config", they are deselected again.

I also get a bunch of messages from googletest about how some kind of deprecation. Might need to update to a newer googletest version or something? But doesn't seem like a serious error so I'll let that one go.

In other news, deactivating the additional libraries at least made the config succeed and I was able to build the projects. Further, I was able to compile the rawspeed subproject but I had to manually remove the "-g0" and "-O3" flags from the compiler settings. Doing that for the single subproject is okay but for the full build of hundreds of projects that might be very obnoxious to do manually. I presume those are gcc-only flags? Clang CL said that -g0 is an unknown argument and that -O3 is an "unused argument", whatever that means.

In any case, I was able to compile a .lib and successfully load it into the linker in my other project. Well, not sure if successfully, but at least the compilation finished, which I'm gonna call a success.

LebedevRI commented 3 years ago

Ah, interesting. Okay so my change would then basically be to remove this part:

CHECK_CXX_COMPILER_FLAG("-std=c++14" COMPILER_SUPPORTS_CXX14)
if(NOT COMPILER_SUPPORTS_CXX14)
  message(FATAL_ERROR "The compiler ${CMAKE_CXX_COMPILER} has no C++14 support. Please use a different C++ compiler.")
else()
  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++14")
  message(STATUS "Checking for -std=c++14 support - works")
endif()

And replace the message about needing both with a short note that that is no longer necessary. Does that make sense? I'm still very much beginning to understand how cmake works so wanna make sure I'm not breaking anything. So far I only commented that part out.

As per, https://gitlab.kitware.com/cmake/cmake/-/commit/f72ba42b7c643b1b217d1b8f6684cec3289e7201#9735efb2f01e32e5293f52150b5e52561787a4ce, i think so.

In other news, deactivating the additional libraries at least made the config succeed and I was able to build the projects. Further, I was able to compile the rawspeed subproject but I had to manually remove the "-g0" and "-O3" flags from the compiler settings.

See above, i have already answered that exact question before.

TomArrow commented 3 years ago

Alright, I'll prepare a PR if/when I find the time. Haven't done one for a while.

Re. the flags, are you referring to your comment about windows-style flags? Bc all the other flags in the VS project are listed with "-" as well and seem to cause no issues, it's only those two he doesn't seem to know what to do with.

WindowsNT commented 2 years ago

Any news on VS cmake? Or perhaps I can manually add the files to a new project?

LebedevRI commented 2 years ago

There are no news because no one has volunteered to do the porting. I would personally strongly suggest to just use clang-cl. Note that if #325 happens, clang will explicitly become the only supported compiler.

WindowsNT commented 2 years ago

99,9% of Visual Studio Windows developers use msvc. The only feasible solution for them therefore would be to build some DLL with rawspeed and even so, this could cause major incompatibilies when two different compilers are used in the same project.

Apart from that, this seems a great library and it's a pity I can't use it as of now, but will perhaps try to work on it later.

LebedevRI commented 2 years ago

99,9% of Visual Studio Windows developers use msvc.

Good for them.