[nettle:x64-windows] build failure

ghost commented 4 years ago

Host Environment

OS: [Microsoft Windows [Version 10.0.19041.331]]
Compiler:
- Visual Studio 2017 Version: 15.9.24 VS Tools Version: 14.16.27023
- Visual Studio 2019 Version: 16.6.3 VS Tools Version: 14.26.28801

To Reproduce Steps to reproduce the behavior: vcpkg install nettle[core]:x64-windows

Failure logs

Cut and past the appropriate build messages from the console output. Computing installation plan... The following packages will be built and installed: nettle[core]:x64-windows Starting package 1/1: nettle:x64-windows Building package nettle[core]:x64-windows... -- Using cached D:/Git/vcpkg.downloads/ShiftMediaProject-nettle-1d0a6e64e01458fdf37eaf5d90975deb52c3da41.tar.gz -- Using source at D:/Git/vs2019.prod/buildtrees/nettle/src/eb52c3da41-e03f56976f -- Building SMP/libnettle.sln for Release CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message): Command failed: msbuild D:/Git/vs2019.prod/buildtrees/nettle/x64-windows-rel/eb52c3da41-e03f56976f/SMP/libnettle.sln /p:Configuration=ReleaseDLL /p:UseEnv=True /t:Rebuild /p:Platform=x64 /p:PlatformToolset=v142 /p:VCPkgLocalAppDataDisabled=true /p:UseIntelMKL=No /p:WindowsTargetPlatformVersion=10.0.19041.0 /p:VcpkgTriplet=x64-windows /p:VcpkgCurrentInstalledDir=D:/Git/vs2019.prod/installed/x64-windows /p:VcpkgManifestInstall=false /m /p:ForceImportBeforeCppTargets=D:/Git/vs2019.prod/scripts/buildsystems/msbuild/vcpkg.targets /p:VcpkgApplocalDeps=false Working Directory: D:/Git/vs2019.prod/buildtrees/nettle/x64-windows-rel/eb52c3da41-e03f56976f Error code: 1 See logs for more information: D:\Git\vs2019.prod\buildtrees\nettle\build-x64-windows-rel-out.log

Call Stack (most recent call first): scripts/cmake/vcpkg_install_msbuild.cmake:162 (vcpkg_execute_required_process) ports/nettle/portfile.cmake:66 (vcpkg_install_msbuild) scripts/ports.cmake:76 (include)

Error: Building package nettle:x64-windows failed with: BUILD_FAILED Please ensure you're using the latest portfiles with.\vcpkg update, then submit an issue at https://github.com/Microsoft/vcpkg/issues including: Package: nettle:x64-windows Vcpkg version: 2020.02.04-nohash

Additionally, attach any relevant sections from the log files above.

Please attach any additional failure logs mentioned in the console output. vs2017-build-x64-windows-rel-out.log vs2019-build-x64-windows-rel-out.log

Additional context This may/may not have relevance from issue [<gmp>] build failure referenced #12237 as I have a work around patch to place the gmp.h and gmpxx.h in the <vcpkg-port>\installed\x64-windows\include folder instead of the <vcpkg-port>\installed\x64-windows\include\include folder.

JackBoosY commented 4 years ago

I know this bug, @Neumann-A could you please take a look?

Thanks.

Neumann-A commented 4 years ago

Probably the segfault in yasm we don't know why it occours. Updating VS somehow resolves the problem.

Neumann-A commented 4 years ago

I am also unable to reproduce the error with Visual Studio 2019 Version: 16.6.3 locally

JackBoosY commented 4 years ago

@Neumann-A Yes, we can't repro this issue with Visual Studio 2019, and this PR is the same as some PR I encountered.

MVoz commented 4 years ago

open ...\x64-windows\share\vs-yasm\yasm.props

  <PropertyGroup>
    <YasmPath Condition= "'$(YASMPATH)' == ''"></YasmPath>
  </PropertyGroup>
  <ItemDefinitionGroup>
    <YASM>
      <Preprocessor>0</Preprocessor>
      <Parser>0</Parser>
      <ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>
      <CommandLineTemplate Condition="'$(Platform)' == 'Win32'">"$(YasmPath)"yasm.exe -Xvc -f win32 [AllOptions] [AdditionalOptions] "%(FullPath)"

"$(YasmPath)"yasm.exe <<--

''"></YasmPath> <<-- null

edit to `

$(VCPKG_ROOT)/download/bla/bla/yasm_path` ____ or delete `"$(YasmPath)"` "$(YasmPath)"yasm.exe -Xvc -f win32 [AllOptions] -->> yasm.exe -Xvc -f win32 [AllOptions] ____ although this does not solve anything, only if explicitly specify the path, there may not be an error

MVoz commented 4 years ago

other error

Spoiler

` Creating library E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\..\msvc\\lib\x64\hogweed.lib and object E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\..\msvc\\lib\x64\hogweed.exp` ``` 4>dsa-sign.obj : error LNK2019: unresolved external symbol mpz_powm_sec referenced in function nettle_dsa_sign [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-blind.obj : error LNK2001: unresolved external symbol mpz_powm_sec [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_mul_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_mul_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_mul referenced in function _nettle_rsa_sec_compute_root [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_mul [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_div_r_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_div_r_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_div_r referenced in function _nettle_rsa_sec_compute_root [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_div_r [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_powm_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_powm_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_powm referenced in function sec_powm [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2001: unresolved external symbol mpn_sec_powm [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_add_1_itch referenced in function _nettle_rsa_sec_compute_root_itch [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sec-compute-root.obj : error LNK2019: unresolved external symbol mpn_sec_add_1 referenced in function _nettle_rsa_sec_compute_root [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2019: unresolved external symbol mpn_sec_invert_itch referenced in function rsa_sec_blind [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj] 4>rsa-sign-tr.obj : error LNK2019: unresolved external symbol mpn_sec_invert referenced in function rsa_sec_blind ``` `E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\..\msvc\\bin\x64\hogweed.dll : fatal error LNK1120: 11 unresolved externals [E:\tools\vcpkg\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\libhogweed.vcxproj]`

MVoz commented 4 years ago

vcpkg remove gmp --triplet x64-windows

vcpkg install gmp --triplet x64-windows

vcpkg install nettle --triplet x64-windows

all ok

MVoz commented 4 years ago

@Neumann-A

have a question for *. pc

gmp.txt gmpxx.txt libhogweed.txt nettle.txt

example 1. libhogweed.pc -> 2. nettle.pc

-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/lib/pkgconfig/libhogweed.pc
-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/lib/pkgconfig/nettle.pc
-- Fixing pkgconfig - debug
-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/debug/lib/pkgconfig/libhogweed.pc
-- Checking file: E:/tools/vcpkg/packages/nettle_x64-windows/debug/lib/pkgconfig/nettle.pc

double space hogweedd <-> nettled Libs: -L${libdir} -lhogweedd -lnettled
separator ; Libs: -L${libdir} -lnettled;-lgmpd

ghost commented 4 years ago

@Voskrese I tried making changes to the <vcpkg-root>\installed\x64-windows\shared\vs-yasm\yasm.props

Deleted "$(YasmPath)"yasm.exe -> yasm.exe failed (yasm.exe is in path a link to yasm.exe in VSTools folder)
Changed to include absolute path "$(YasmPath)"yasm.exe -> D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\yasm.exe
set YASMPATH=D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\ and include it in vcpkg env passthrough
I tried many other ways this and that spent most of the morning on this... :(

All seem to fail with about the same issue with YASM.

I will just figure something else out, especially since @JackBoosY and others can not repro it on their instances :(

MVoz commented 4 years ago

test?

<YasmPath Condition= "'$(YASMPATH)' == ''">D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\</YasmPath>

ghost commented 4 years ago

Ok made change from: <PropertyGroup> <YasmPath Condition= "'$(YASMPATH)' == ''"></YasmPath> </PropertyGroup> to: <PropertyGroup> <YasmPath Condition= "'$(YASMPATH)' == ''">D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\</YasmPath> </PropertyGroup>

Now it states no input file :( _YASM: x86_64\aes-decrypt-internal-2-fat.s cmd.exe /D /C "S:\TEMP\vs2019.test\tmp14a20c86af284625a141ba926d321515.cmd" "D:\Git\vcpkg.downloads\tools\yasm\1.3.0.6\"yasm.exe -Xvc -f win64 -o "D:\Git\vs2019.test\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\obj\ReleaseDLL\x64\libnettle\aes-decrypt-internal-2-fat.obj" -rraw -pgas "D:\Git\vs2019.test\buildtrees\nettle\x64-windows-rel\eb52c3da41-e03f56976f\SMP\x86_64\aes-decrypt-internal-2-fat.s" yasm: No input files specified

I think I take a break on this one for a while, I am getting burnt out on it. I will look fresh later on. Thank you very much for help so far - I am learning some new things, I appreciate the lessons. :)

Neumann-A commented 4 years ago

@Voskrese

separator ;

Ah thats an error in the portfile. set(LIBS -lnettle -lgmp) should probably be set(LIBS "-lnettle -lgmp")

double space hogweedd <-> nettled

doesn't matter. I also do not patch anything of this stuff (just configure it.)

Sibras commented 4 years ago

FYI The 1.3.0 release of YASM has several bugs that cause segfaults and memory leaks when using the gas parser and COFF outputs. This is what is used by nettle and some other ports. So you may want to update the yasm port to instead pull in yasm master instead of the old 1.3.0 tag. This will fix some known issues, as to whether it fixes this issue i dont know.

MVoz commented 4 years ago

@LilyWangL https://github.com/microsoft/vcpkg/tree/master/ports/yasm

The 1.3.0 release of YASM has several bugs that cause segfaults and memory leaks

update + install tools ?

MVoz commented 4 years ago

@Neumann-A

set(LIBS "-lnettle -lgmp")

yes, this corrected the situation however, I don't know if the separator ; will be the cause of the error in the library search

ghost commented 4 years ago

@Voskrese cc: @JackBoosY @LilyWangL @Neumann-A @Sibras Ok - I finally got nettle installed - here is what I did:

Created a new CONTROL and portfile.cmake for yasm
- CONTROL file change was replacing the version 1.3.0 with the lastest commit date at github.com/yasm 2020-05-04
- portfile.cmake changed the REF to latest commit id and modified the SHA512 to match. vcpkg_from_github( OUT_SOURCE_PATH SOURCE_PATH REPO yasm/yasm REF c9db6d7 SHA512 cbc083f80e61b898ee1eee99baa2c684fb5c928c8ceaa2dcd2e72e49fae8a1a86432df59e509abdec62343d989603440fa68b7996a25620dbc4303de0e295d02 HEAD_REF master )
Using Microsoft Windows [Version 10.0.19041.331] Visual Studio 2019 Version: 16.6.3 VS Tools Version: 14.26.28801
.\vcpkg.exe install yasm[core]:x64-windows-static
Copied the created 64-bit static yasm.exe to <vcpkg-root>\downloads\tools\yasm\yasm-1.3.0 replacing the 32-bit downloaded version by vcpkg.
.\vcpkg.exe install nettle[core]:x64-windows - installed fine after making the one and only change (mention in 5 below) to gmp and nettle portfile.cmake file.
I also had an issue with both gmp and nettle, where I had to change the following line:
- from file(RENAME "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/${SOURCE_PATH_SUFFIX}/msvc/include" "${CURRENT_PACKAGES_DIR}/include")
- to file(INSTALL "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/${SOURCE_PATH_SUFFIX}/msvc/include" DESTINATION "${CURRENT_PACKAGES_DIR}")
- as it tells me trying to rename <vcpkg-root>\<port>\<triplet>-rel\<src-path>\msvc\include to <vcpkg-root>\packages\<port>_<triplet>\include already exists
- but in reality it appears it is trying to rename it to <vcpkg-root>\installed\<triplet>\include\<port>
- in essence I am not sure this is the correct use of cmake RENAME since the include directory/folder already exists and the portfile.cmake should have an cmake INSTALL instead? or could it be cmake version differences as I use cmake 3.17.3?
I have noticed others getting similar errors in some of the issue posts mentioning rename failures and I am wondering if this is the same issue?

MVoz commented 4 years ago

@Neumann-A

tested in http://www.tortall.net/projects/yasm/releases/yasm-1.3.0-win64.exe http://www.tortall.net/projects/yasm/releases/yasm-1.3.0-win32.exe

yasm x86 + props vs = error build x64

moy patch

vcpkg_find_acquire_program.cmake

...
    if(VCPKG_TARGET_ARCHITECTURE STREQUAL x86)
        set(URL "https://www.tortall.net/projects/yasm/snapshots/v1.3.0.6.g1962/yasm-1.3.0.6.g1962.exe")
        set(ARCHIVE "yasm-1.3.0.6-win32.exe")
        set(_vfa_RENAME "yasm.exe")
        set(NOEXTRACT ON)
        set(HASH c1945669d983b632a10c5ff31e86d6ecbff143c3d8b2c433c0d3d18f84356d2b351f71ac05fd44e5403651b00c31db0d14615d7f9a6ecce5750438d37105c55b)
    else(VCPKG_TARGET_ARCHITECTURE STREQUAL x64)
        set(URL "https://github.com/yasm/yasm/releases/download/v1.3.0/yasm-1.3.0-win64.exe")
        set(ARCHIVE "yasm-1.3.0-win64.exe")
        set(_vfa_RENAME "yasm.exe")
        set(NOEXTRACT ON)
        set(HASH 73dfd4ccf93972bb6e4794c071b712de0dbffe6d10345fd9d0b0a9c2472d87cd06f6ac32770af9ba2abb715ad0f80e2a55cf02284f44627dc5e303d66065336f)
    endif()
    get_filename_component(YASM_DIR "${PATHS}/${SUBDIR}" DIRECTORY)
    vcpkg_add_to_path(PREPEND "${YASM_DIR}")
    set(YASMPATH "${YASM_DIR}")
...

portfile.cmake

vcpkg_find_acquire_program(YASM) #<<-- can leave only one line

#    get_filename_component(YASM_DIR "${YASM}" DIRECTORY)
#    vcpkg_add_to_path("${YASM_DIR}")

#          /p:UseEnv=True

MVoz commented 4 years ago

OR VS PROPS

portfile.cmake

    vcpkg_find_acquire_program(YASM)
    get_filename_component(YASM_DIR "${YASM}" DIRECTORY)
    vcpkg_add_to_path("${YASM_DIR}")
...
        OPTIONS
#          /p:UseEnv=True
+          "/p:YASMPATH=${YASM_DIR}/"

OR VS PROPS

portfile.cmake

    vcpkg_find_acquire_program(YASM)
    get_filename_component(YASM_DIR "${YASM}" DIRECTORY)
    vcpkg_add_to_path("${YASM_DIR}")
...
#        OPTIONS
#          /p:UseEnv=True

vcpkg_install_msbuild.cmake

...
    if(NOT DEFINED _csc_YASMPATH)
        set(_csc_YASMPATH ${YASM_DIR})
    endif()
...
...
    if(DEFINED _csc_YASMPATH)
        list(APPEND _csc_OPTIONS
            "/p:YASMPATH=${YASM_DIR}/"
            )
    endif()

JackBoosY commented 4 years ago

Related: #10613.

JackBoosY commented 4 years ago

@Voskrese Sadly, none of the solutions you provide can be used in Visual Studio 2017.

Neumann-A commented 4 years ago

Hmm why not? I thought the solution would be to just download the correct architecture build of yasm? So just changing vcpkg_find_acquire_program

JackBoosY commented 4 years ago

After changing the yasm binary, seems work. I'm fixing this issue now.

MVoz commented 4 years ago

Sadly, none of the solutions you provide can be used in Visual Studio 2017.

this case is voluntary to use or not, my business is to offer and it works on MSVC 2013 b MSVC 2015 on account of 2017, I do not know, do not use

ghost commented 3 years ago

I close, I no longer use product.

microsoft / vcpkg

[nettle:x64-windows] build failure #12240