search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.45k stars 3.52k forks source link

[C++][Gandiva] Solve LLVM 15 CI performace and compatibility issues #20439

Open asfimport opened 2 years ago

asfimport commented 2 years ago

Package managers have been upgrading their default LLVM version to 15. This raises several compatibility and performance issues for CI, and we currently are using workaounds

  1. Mac's default clang 13's LTO cannot read Brew installed llvm static libs because they are compiled with llvm 15 (https://github.com/apache/arrow/pull/14187#issuecomment-1253322948). Workaround: Use brew installed clang: ARROW-17805 ARROW-17842
  2. Conda's packaging of LLVM 15 on Windows is not portable. It required a VS2019 Enterprise version: https://github.com/conda-forge/llvmdev-feedstock/issues/175. Workaround: Pin LLVM to 14 for Conda with Windows. ARROW-17830
  3. Brew's LLVM15 enables z3 by default and it's shared linked. It breaks Java jar builds on MacOS: https://github.com/apache/arrow/pull/14261#issuecomment-1263024112. Workaround: Pin LLVM to 14 for all MacOS tasks.

  4. LLVM 15 is noticably slower to build than LLVM 14 for brew and MacOS. Workaround: Pin LLVM to 14 for all MacOS tasks. ARROW-17872

    We need to find a way to resolve these workarounds for LLVM 15.

    Edit: The current mac os runner we use macos-latest has llvm@14 preinstalled. So pinning to 14 can save build time and fix dependency issues. We can move to llvm@15 on mac once the github images support them by default.

Reporter: Jin Shang / @js8544

Related issues:

Note: This issue was originally created as ARROW-17902. Please see the migration documentation for further details.

asfimport commented 1 year ago

Apache Arrow JIRA Bot: This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.