Duplicate symbol when compiling with clang-cl

ValeV008 commented 1 year ago

Description

Building static version with compiler clang-cl (and its linker lld-link) leads to linking errors. Was mentioned here, but not solved. If @dewyatt could share the fix mentioned in that comment, that would be good.

System: Windows 10 64 bit, cmake.

Steps to Reproduce

cmake .. -D CMAKE_CONFIGURATION_TYPES=Release -D BUILD_SHARED_LIBS=off -D BUILD_TESTING=off -D CRYPTO_BACKEND="openssl" -D ENABLE_COVERAGE=off -G "Visual Studio 16 2019" -T ClangCL -A x64
cmake --build . --config Release

Expected Behavior

Successful build.

Actual Behavior

Lots of "lld-link : error : duplicate symbol ..." errors.

ronaldtse commented 1 year ago

Thank you @ValeV008 for raising this issue proper!

@maxirmx do you have time to investigate this?

maxirmx commented 1 year ago

Thank you, @ValeV008 I will look at it

maxirmx commented 1 year ago

@ValeV008 It looks like you used several configuration options that we never tested on Windows I believe I understand these two:

openssl backend (#2021)
llvm toolset (#2022)

I need some clarification regarding some other matters:

what version of rnp did you build ?
we manage external dependencies (openssl, json-c, bzip2, zlib) using vcpkg How did you manage dependencies in your run ?
you wrote 'Building static version ... leads to linking errors.' I am afraid there may be some confusion. We always build static version. If -D BUILD_SHARED_LIBS=off we do not build dynamic library, that's it. I will be happy to discuss available options if you explain me your objectives

Thank you

ValeV008 commented 1 year ago

@maxirmx Hi, thanks for the reply.

I git cloned version v0.16.2
I git cloned the dependencies json-c, openssl, zlib and bzip, and compiled them by hand (using cmake with clang-cl). For the getopt and ident i used vcpkg. When running cmake on rnp (the "steps to reproduce" in first comment), i included the paths to the dependencies.
I see, I thought "BUILD_SHARED_LIBS=on" option builds .dll file, while "BUILD_SHARED_LIBS=off" builds .lib file.

I imagine I only need 2 builds to succeed to get all the files (static and dynamic libs for release and debug). One with BUILD_SHARED_LIBS=on and CMAKE_CONFIGURATION_TYPES=Release and one with BUILD_SHARED_LIBS=on and CMAKE_CONFIGURATION_TYPES=Debug

Funny thing is, building rnp with same commands as in first comment, but with "BUILD_SHARED_LIBS=on", was successful. If I also change CMAKE_CONFIGURATION_TYPES to debug, I get compiler errors, but this is for another thread.

ValeV008 commented 1 year ago

diff.patch

@maxirmx I completely forgot that I was changing rnp source code as well. Here is diff patch showing what I've changed.

maxirmx commented 1 year ago

diff.patch

@maxirmx I completely forgot that I was changing rnp source code as well. Here is diff patch showing what I've changed.

Thank you, although it is a little bit outdated :) Actually, you were one step from success. I think if you fork from https://github.com/rnpgp/rnp/tree/maxirmx-2020-clang it will work for you (please clone with submodules) I cannot make it run on github but it worls locally both with cl and openssl

maxirmx commented 1 year ago

3. I see, I thought "BUILD_SHARED_LIBS=on" option builds .dll file, while "BUILD_SHARED_LIBS=off" builds .lib file.

In 0.16.2 BUILD_SHARED_LIBS=off builds static lib and makes it public BUILD_SHARED_LIBS=on builds static lib, builds dynamic lib on top of static and makes dynamic lib public

ValeV008 commented 1 year ago

diff.patch @maxirmx I completely forgot that I was changing rnp source code as well. Here is diff patch showing what I've changed.

Thank you, although it is a little bit outdated :) Actually, you were one step from success. I think if you fork from https://github.com/rnpgp/rnp/tree/maxirmx-2020-clang it will work for you (please clone with submodules) I cannot make it run on github but it worls locally both with cl and openssl

I cloned the branch and init updated submodoles. When doing cmake configure, I got:

` C:\Users\C081745\Desktop\rnp_offbranch\build>cmake ..\rnp -D CMAKE_CONFIGURATION_TYPES=Release -D CMAKE_INSTALL_PREFIX="INSTALL" -D BUILD_SHARED_LIBS=on -D BUILD_TESTING=off -D CRYPTO_BACKEND="openssl" -D ENABLE_COVERAGE=off -D DIRENT_INCLUDE_DIR="C:/Users/C081745/Desktop/rnp/include" -D GETOPT_INCLUDE_DIR="C:/Users/C081745/Desktop/rnp/getopt" -D GETOPT_LIBRARY="C:/Users/C081745/Desktop/rnp/getopt/getopt.lib" -D CMAKE_PREFIX_PATH="C:\Users\C081745\Desktop\tplib64_clang_x64\openssl;C:\Users\C081745\Desktop\tplib64_clang_x64\zlib;C:\Users\C081745\Desktop\bzip2;C:\Users\C081745\Desktop\json-c_all\build\INSTALL" -G "Visual Studio 16 2019" -T ClangCL -A x64 -- Found version.txt with 0.16.2 -- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.22.0.windows.1") -- Found no annotated tags. -- RNP_VERSION: 0.16.2 -- RNP_VERSION_NCOMMITS: 0 -- RNP_VERSION_GIT_REV: f4bb655 -- RNP_VERSION_IS_DIRTY: FALSE -- RNP_VERSION_COMMIT_TIMESTAMP: 1679997932 -- RNP_VERSION_SUFFIX: +git20230328.f4bb655 -- RNP_VERSION_FULL: 0.16.2+git20230328.f4bb655 -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044. -- The C compiler identification is Clang 11.0.0 with MSVC-like command-line -- The CXX compiler identification is Clang 11.0.0 with MSVC-like command-line -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/Llvm/x64/bin/clang-cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/Llvm/x64/bin/clang-cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done CMake Warning at cmake/Modules/AdocMan.cmake:51 (message): AsciiDoc processor not found, man pages will not be generated. Install asciidoctor or use the CMAKE_PROGRAM_PATH variable. Call Stack (most recent call first): CMakeLists.txt:180 (include)

-- Found version.txt with 0.8.2 -- Found no annotated tags. -- SEXP_VERSION: 0.8.2 -- SEXP_VERSION_NCOMMITS: 0 -- SEXP_VERSION_GIT_REV: ee796f5 -- SEXP_VERSION_IS_DIRTY: FALSE -- SEXP_VERSION_COMMIT_TIMESTAMP: 1679721435 -- SEXP_VERSION_SUFFIX: +git20230328.ee796f5 -- SEXP_VERSION_FULL: 0.8.2+git20230328.ee796f5 -- Building Release configuration -- Found BZip2: optimized;C:/Users/C081745/Desktop/bzip2/libbz2.lib;debug;C:/Users/C081745/Desktop/bzip2/libbz2d.lib (found version "1.0.8") -- Looking for BZ2_bzCompressInit -- Looking for BZ2_bzCompressInit - found -- Found ZLIB: C:/Users/C081745/Desktop/tplib64_clang_x64/zlib/lib/zlib.lib (found version "1.2.12") -- Found JSON-C: C:/Users/C081745/Desktop/json-c_all/build/INSTALL/lib/json-c.lib (found suitable version "0.16", minimum required is "0.11") -- Found OpenSSL: C:/Users/C081745/Desktop/tplib64_clang_x64/openssl/lib/libcrypto.lib (found version "3.0.5") -- Found OpenSSL: C:/Users/C081745/Desktop/tplib64_clang_x64/openssl/lib/libcrypto.lib (found suitable version "3.0.5", minimum required is "1.1.1") CMake Error at cmake/Modules/FindOpenSSLFeatures.cmake:97 (message): Error configuring findopensslfeatures: 1

CMake Error at CMakeLists.txt:2 (project):

Failed to run MSBuild command:

  C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/MSBuild/Current/Bin/MSBuild.exe

to get the value of VCTargetsPath:

  Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework
  Copyright (C) Microsoft Corporation. All rights reserved.

  Build started 28/03/2023 14:09:30.
  Project "C:\Users\C081745\Desktop\rnp_offbranch\build\fossl\build\CMakeFiles\3.18.4\VCTargetsPath.vcxproj" on node 1 (default targets).
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(816,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform=' x64'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\Users\C081745\Desktop\rnp_offbranch\build\fossl\build\CMakeFiles\3.18.4\VCTargetsPath.vcxproj]
  Done Building Project "C:\Users\C081745\Desktop\rnp_offbranch\build\fossl\build\CMakeFiles\3.18.4\VCTargetsPath.vcxproj" (default targets) -- FAILED.

  Build FAILED.

  "C:\Users\C081745\Desktop\rnp_offbranch\build\fossl\build\CMakeFiles\3.18.4\VCTargetsPath.vcxproj" (default target) (1) ->
  (_CheckForInvalidConfigurationAndPlatform target) ->
    C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(816,5): error : The BaseOutputPath/OutputPath property is not set for project 'VCTargetsPath.vcxproj'.  Please check to make sure that you have specified a valid combination of Configuration and Platform for this project.  Configuration='Debug'  Platform=' x64'.  You may be seeing this message because you are trying to build a project without a solution file, and have specified a non-default Configuration or Platform that doesn't exist for this project. [C:\Users\C081745\Desktop\rnp_offbranch\build\fossl\build\CMakeFiles\3.18.4\VCTargetsPath.vcxproj]

      0 Warning(s)
      1 Error(s)

  Time Elapsed 00:00:00.15

Exit code: 1

Call Stack (most recent call first): src/lib/CMakeLists.txt:40 (include)

-- Configuring incomplete, errors occurred! See also "C:/Users/C081745/Desktop/rnp_offbranch/build/CMakeFiles/CMakeOutput.log". `

maxirmx commented 1 year ago

Hi @ValeV008 We now have a version that works on Windows/openssl/cl I was not able to reproduce your last issue though I have created a test that was very close to your config. I also added some debug output so I will appreciate if you try this branch: https://github.com/rnpgp/rnp/tree/maxirmx-2020-2

rnpgp / rnp