search

intelxed / xed

The X86 Encoder Decoder (XED), is a software library for encoding and decoding X86 (IA32 and Intel64) instructions
https://intelxed.github.io/
Apache License 2.0
1.41k stars 148 forks source link

build win32 version error #196

Open eaglesharkmayonnaise opened 5 years ago

eaglesharkmayonnaise commented 5 years ago 
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.9.13
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x86'

run :

mfile.py host_cpu=ia32


[BINDING] host_cpu --> [ia32]
[MBUILD WARNING] Could not find 32b RC command in SDK directory; assuming on PATH
[PYTHON VERSION] 2.7.15
[GIT VERSION] v10.0-411-gb7231de
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/files-xregs.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/files-via-padlock.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/files-amd.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/amdxop/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/mpx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/cet/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/rdrand/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/glm/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/sha/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/xsaveopt/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/xsaves/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/xsavec/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/clflushopt/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/rdseed/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/fsgsbase/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/smap/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/sgx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/rdpid/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/pt/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/tremont/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/movdir/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/waitpkg/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/cldemote/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/sgx-enclv/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx/files.cfg
[Clearing file list for type dec-spine: [ C:/Users/kkk/Desktop/xedtes/xed/datafiles/xed-spine.txt ]]
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/ivbavx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/hswavx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/hswbmi/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/hsw/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/bdw/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/skl/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/skx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/pku/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/clwb/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/clx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/vnni/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/knl/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/knm/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/4fmaps-512/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/4vnniw-512/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/vpopcntdq-512/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx512f/shared-files.cfg
[Clearing file list for type dec-spine: [ C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx/avx-spine.txt ]]
CONSIDERING SOURCE C:\Users\kkk\Desktop\xedtes\xed\datafiles\knc\xed-operand-values-interface-uisa.c source 1
ADDING SOURCE C:\Users\kkk\Desktop\xedtes\xed\datafiles\knc\xed-operand-values-interface-uisa.c source 1
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx512f/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx512cd/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx512-skx/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/cnl/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx512ifma/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/avx512vbmi/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/icl/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/wbnoinvd/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/pconfig/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/bitalg/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/vbmi2/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/gfni-vaes-vpcl/files-sse.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/gfni-vaes-vpcl/files-avx-avx512.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/vpopcntdq-vl/files.cfg
[EXTF PROCESSING] C:/Users/kkk/Desktop/xedtes/xed/datafiles/future/files.cfg
[EMIT BUILD DEFINES HEADER FILE]
R: 1 P: 0 C: 0 E: 0 / 12 msecs [decprep]
[TOUCH] obj/dummy-prep
R: 0 P: 0 C: 1 E: 0 / 690 msecs
BUILT: C:\Users\kkk\Desktop\xedtes\xed\obj\dummy-prep
R: 2 P: 0 C: 1 E: 0 / 690 msecs [encgen decgen]
[WRITING] obj/ENC-OUT.txt
[WRITING] obj/ENC-ERR.txt
[ENC-GEN] Return code: 0
R: 1 P: 0 C: 2 E: 0 / 16 secs [decgen]
BUILT: C:\Users\kkk\Desktop\xedtes\xed\obj\ENCGEN-OUTPUT-FILES.txt
[WRITING] obj/DEC-OUT.txt
[WRITING] obj/DEC-ERR.txt
[DEC-GEN] Return code: 0
R: 0 P: 0 C: 3 E: 0 / 16 secs
BUILT: C:\Users\kkk\Desktop\xedtes\xed\obj\DECGEN-OUTPUT-FILES.txt
R: 4 P: 107 C: 0 E: 0 / 15 msecs
[MBUILD WARNING] Command execution failed. Waiting for remaining jobs and exiting.
R: 3 P: 107 C: 1 E: 1 / 186 msecs
[COMMAND     ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private     /nologo  /MT  /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-rep-map.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed-rep-map.c
[EXIT_STATUS ] 1
[STDERR]
Execution error for: 'utf8' codec can't decode byte 0xce in position 23: invalid continuation byte
[COMMAND     ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private     /nologo  /MT  /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-rep-map.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed-rep-map.c
[EXIT_STATUS ] 1

[MBUILD WARNING] Command execution failed. Waiting for remaining jobs and exiting. R: 2 P: 107 C: 2 E: 2 / 188 msecs [COMMAND ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private /nologo /MT /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-operand-convert-init.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed-operand-convert-init.c [EXIT_STATUS ] 1 [STDERR] Execution error for: 'utf8' codec can't decode byte 0xce in position 23: invalid continuation byte [COMMAND ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private /nologo /MT /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-operand-convert-init.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed-operand-convert-init.c [EXIT_STATUS ] 1

[MBUILD WARNING] Command execution failed. Waiting for remaining jobs and exiting. R: 1 P: 107 C: 3 E: 3 / 190 msecs [COMMAND ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private /nologo /MT /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-isa-set-enum.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed-isa-set-enum.c [EXIT_STATUS ] 1 [STDERR] Execution error for: 'utf8' codec can't decode byte 0xce in position 23: invalid continuation byte [COMMAND ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private /nologo /MT /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed-isa-set-enum.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed-isa-set-enum.c [EXIT_STATUS ] 1

[MBUILD WARNING] Command execution failed. Waiting for remaining jobs and exiting. R: 0 P: 107 C: 4 E: 4 / 191 msecs [COMMAND ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private /nologo /MT /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed3-phash-lu-vv1.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed3-phash-lu-vv1.c [EXIT_STATUS ] 1 [STDERR] Execution error for: 'utf8' codec can't decode byte 0xce in position 23: invalid continuation byte [COMMAND ] "C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023//bin/Hostx64/x86/cl.exe" -IC:/Users/kkk/Desktop/xedtes/xed/include/private -IC:/Users/kkk/Desktop/xedtes/xed/include/public/xed -IC:/Users/kkk/Desktop/xedtes/xed/include/public -Iobj -Iobj/include-private /nologo /MT /W4 /WX /wd4091 /wd4127 /wd4505 /wd4702 /wd4244 /wd4292 /DXED_GIT_VERSION="v10.0-411-gb7231de" /DXED_AMD_ENABLED /DXED_VIA_ENABLED /DXED_AVX /DXED_SUPPORTS_AVX512 /DXED_MPX /DXED_CET /DXED_SUPPORTS_SHA /DXED_SUPPORTS_WBNOINVD /DXED_DECODER /DXED_ENCODER /DXED_SUPPORTS_LZCNT_TZCNT /DXED_BUILD /c /Foobj/xed3-phash-lu-vv1.obj C:/Users/kkk/Desktop/xedtes/xed/obj/xed3-phash-lu-vv1.c [EXIT_STATUS ] 1

ABORT: Library build failed

markcharney commented 5 years ago

that is an odd unicode error. what tools did you use to check out the sources? did you edit the sources? as a workaround to see if it helps can you try installing https://gitforwindows.org/ and checking out the sources with that git?

eaglesharkmayonnaise commented 5 years ago

I didn't edit anything.

markcharney commented 5 years ago

didn’t understand your last statement.

i have had many people build on windows. i am guessing it is something about your configuration.

markcharney commented 5 years ago

try https://gitforwindows.org/ as an experiment...

what git are you using?

eaglesharkmayonnaise commented 5 years ago

i use this: https://git-scm.com/download/win

flobernd commented 5 years ago

I had this problem as well, but it disappeared somehow. There was a problem with the SDK pathes as well, but this batch file works fine for me since then:

cd /D "D:\Development\xed\Build64"
set WindowsSdkDir=
python ../mfile.py clean
python ../mfile.py examples
pause

It's even working when executed outside the Developer Command Prompt (maybe VS 2019 automatically added some stuff to the PATH).

markcharney commented 5 years ago

If someone can give me the exact recipe that generates the issue, I can try to reproduce it locally.

OZ1 commented 5 years ago

chcp 65001 set VSLANG=1033

OZ1 commented 5 years ago

If someone can give me the exact recipe that generates the issue, I can try to reproduce it locally.

Install non-english Visual Studio. My Cyrillic installation emits errors in 866 code page by default.

markcharney commented 5 years ago

Can you run this command and tell me what it returns:

python3 -c "import locale; print(locale.getpreferredencoding())"

or python2 instead of python3.

I have a guess that mbuild's hardcoded use of utf-8 is problematic for some languages. Easy to change (I have a patch locally), just wanted to get more data for my hunch

OZ1 commented 5 years ago

C:\VS>python3 -c "import locale; print(locale.getpreferredencoding())" "python3" не является внутренней или внешней командой, исполняемой программой или пакетным файлом.

C:\VS>python -c "import locale; print(locale.getpreferredencoding())" cp1251

C:\VS>where python C:\Python37\python.exe C:\Python27\python.exe

markcharney commented 4 years ago

so I pushed a patch to the mbuild git repo that might address this issue. Please give it a shot and let me know.

OZ1 commented 4 years ago

works for me with no hacks applied

markcharney commented 4 years ago

cool. (And my dreams of getting a job as unicode consultant continue to thrive :-) )