search

intel / hdk

A low-level execution library for analytic data processing.
Apache License 2.0
31 stars 14 forks source link

Enable HDK on Windows #31

Open ienkovich opened 2 years ago

ienkovich commented 2 years ago

Enabling includes successful execution of all OmniSci and HDK tests and integration with Modin.

alexbaden commented 2 years ago

Blocked by problems w/ the cython build:

running build_ext
building 'pyhdk._common' extension
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DBOOST_LOG_DYN_LINK=1 -DWIN_32=1 -IC:\Users\alexb\miniconda3\envs\omnisci-dev\lib\site-packages\numpy\core\include -IC:\Users\alexb\miniconda3\envs\omnisci-dev\lib\site-packages\pyarrow\include -IC:/Users/alexb/Projects/hdk/python -IC:/Users/alexb/Projects/hdk -IC:/Users/alexb/Projects/hdk/omniscidb -IC:/Users/alexb/Projects/hdk/omniscidb/ThirdParty -IC:/Users/alexb/Projects/hdk/omniscidb/ThirdParty/rapidjson -IC:/Users/alexb/miniconda3/envs/omnisci-dev/Library/include -IC:/Users/alexb/Projects/hdk/python -IC:\Users\alexb\miniconda3\envs\omnisci-dev\include -IC:\Users\alexb\miniconda3\envs\omnisci-dev\include /EHsc /TpC:/Users/alexb/Projects/hdk/python/pyhdk/_common.cpp /Fobuild\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk/_common.obj /std:c++17
_common.cpp
C:\Users\alexb\Projects\hdk\omniscidb\QueryEngine\Descriptors/CountDistinctDescriptor.h(56): warning C4334: '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29910\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\lib\site-packages\pyarrow /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\Library\lib /LIBPATH:C:/Users/alexb/Projects/hdk/build/python /LIBPATH:. /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/Calcite/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/ArrowStorage/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/QueryEngine/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/DataMgr/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/SchemaMgr/Release /LIBPATH:C:/Users/alexb/Projects/hdk/build/omniscidb/Shared/Release /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\libs /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\libs /LIBPATH:C:\Users\alexb\miniconda3\envs\omnisci-dev\PCbuild\amd64 arrow.lib arrow_python.lib QueryEngine.lib Shared.lib /EXPORT:PyInit__common build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk/_common.obj /OUT:build\lib.win32-3.8\pyhdk\_common.cp38-win_amd64.pyd /IMPLIB:build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk\_common.cp38-win_amd64.lib
   Creating library build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk\_common.cp38-win_amd64.lib and object build\temp.win32-3.8\Release\Users/alexb/Projects/hdk/python/pyhdk\_common.cp38-win_amd64.exp
Shared.lib(DateTimeParser.obj) : error LNK2001: unresolved external symbol __CxxFrameHandler4
QueryEngine.lib(WindowFunctionIR.obj) : error LNK2001: unresolved external symbol __CxxFrameHandler4
QueryEngine.lib(RexSubQueryIdCollector.obj) : error LNK2001: unresolved external symbol __CxxFrameHandler4

repeats

I am using the same x86_x64 cross compiler for both cmake and cython, so am not sure where this mismatch is coming from. We do not appear to have any control over the Cython internals.

I am currently trying this on Windows 11, which may be part of the problem. But if our delivery mechanism is going to be conda, perhaps we should be trying mingw instead of msvc?

alexbaden commented 2 years ago

Visual Studio 2022 build tools fails in compilation of the library:

C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.32.31326\include\yvals_core.h(599,2): error G69703BD3: STL1000: Unexpected compiler version, expected Clang 13.0.0 or newer.

Attempting a conda solve with clang-13.

alexbaden commented 2 years ago

Same issue with latest msvc and llvm-13. Going to pursue MinGW.

aregm commented 1 year ago

https://github.com/conda-forge/heavydb-ext-feedstock is functional - need to use this for enabling Windows build.

alexbaden commented 1 year ago

I don't think that's particularly useful given divergence in dependencies and code bases. What were you hoping to learn from it?

Garra1980 commented 1 year ago

I don't think that's particularly useful given divergence in dependencies and code bases. What were you hoping to learn from it?

+1 I doubt it will help hdk enabling

aregm commented 1 year ago

Ok. Then we're back to square 1.