heavyai / rbc

Remote Backend Compiler
https://rbc.readthedocs.io
BSD 3-Clause "New" or "Revised" License
30 stars 10 forks source link

Unable to run tests #492

Open bmanturner opened 2 years ago

bmanturner commented 2 years ago

I ran pytest -rs -sv rbc/tests/heavydb/test_array.py -x --tb=short

and received:

rbc/tests/heavydb/test_array.py::test_ptr[cpu-int8_t i1]
  /Users/bfturner/dev/omni/rbc/rbc/irtools.py:130: UserWarning: JITRemoteCodegen._get_host_cpu_features: `remove_features` dictionary requires an update: detected different LLVM versions in server (9, 0, 1) and client (10, 0, 1). CPU features: +sse2,+cx16,+sahf,-tbm,-avx512ifma,-sha,-gfni,-fma4,-vpclmulqdq,+prfchw,+bmi2,-cldemote,+fsgsbase,-ptwrite,-xsavec,+popcnt,-mpx,+aes,-avx512bitalg,-movdiri,-xsaves,-avx512er,-avx512vnni,-avx512vpopcntdq,-pconfig,-clwb,-avx512f,-clzero,-pku,+mmx,-lwp,-rdpid,-xop,-rdseed,-waitpkg,-movdir64b,-sse4a,-avx512bw,-clflushopt,+xsave,-avx512vbmi2,+64bit,-avx512vl,-invpcid,-avx512cd,+avx,-vaes,+cx8,+fma,-rtm,+bmi,-enqcmd,+rdrnd,-mwaitx,+sse4.1,+sse4.2,+avx2,+fxsr,-wbnoinvd,+sse,+lzcnt,+pclmul,-prefetchwt1,+f16c,+ssse3,-sgx,-shstk,+cmov,-avx512vbmi,-avx512bf16,+movbe,+xsaveopt,-avx512dq,-adx,-avx512pf,+sse3.
    f'{type(self).__name__}._get_host_cpu_features: `remove_features` dictionary'

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

so I added the following to remove_features dictionary

(9, 10): ['sse2', 'cx16', 'sahf', 'tbm', 'avx512ifma', 'sha', 'gfni',
                      'fma4', 'vpclmulqdq', 'prfchw', 'bmi2', 'cldemote', 'fsgsbase',
                      'ptwrite', 'xsavec', 'popcnt', 'mpx', 'aes', 'avx512bitalg',
                      'movdiri', 'xsaves', 'avx512er', 'avx512vnni', 'avx512vpopcntdq',
                      'pconfig', 'clwb', 'avx512f', 'clzero', 'pku', 'mmx', 'lwp',
                      'rdpid', 'xop', 'rdseed', 'waitpkg', 'movdir64b', 'sse4a',
                      'avx512bw', 'clflushopt', 'xsave', 'avx512vbmi2', '64bit',
                      'avx512vl', 'invpcid', 'avx512cd', 'avx', 'vaes', 'cx8', 'fma',
                      'rtm', 'bmi', 'enqcmd', 'rdrnd', 'mwaitx', 'sse4.1', 'sse4.2',
                      'avx2', 'fxsr', 'wbnoinvd', 'sse', 'lzcnt', 'pclmul', 'prefetchwt1',
                      'f16c', 'ssse3', 'sgx', 'shstk', 'cmov', 'avx512vbmi', 'avx512bf16',
                      'movbe', 'xsaveopt', 'avx512dq', 'adx', 'avx512pf', 'sse3'],

and now I receive

test_ptr[cpu-int8_t i1] ________________________________________________________________________________________________
rbc/tests/heavydb/test_array.py:105: in test_ptr
    f'select {cname}, mysum_ptr({cname}) from {heavydb.table_name}')
rbc/heavydb/remoteheavydb.py:816: in sql_execute
    self.register()
rbc/heavydb/remoteheavydb.py:1153: in register
    return self._register()
rbc/heavydb/remoteheavydb.py:1218: in _register
    debug=self.debug)
rbc/irtools.py:407: in compile_to_LLVM
    user_defined_llvm_ir = llvm.parse_assembly(user_defined_llvm_ir)
../../../omnisci-datascience/lib/python3.7/site-packages/llvmlite/binding/module.py:25: in parse_assembly
    raise RuntimeError("LLVM IR parsing error\n{0}".format(errmsg))
E   RuntimeError: LLVM IR parsing error
E   <string>:58:19: error: unterminated attribute group
E   attributes #0 = { mustprogress noinline nounwind optnone ssp uwtable "darwin-stkchk-strong-link" "frame-pointer"="all" "min-legal-vector-width"="0" "no-trapping-math"="true" "probe-stack"="___chkstk_darwin" "stack-protector-buffer-size"="8" "target-cpu"="penryn" "target-features"="+cx16,+cx8,+fxsr,+mmx,+sahf,+sse,+sse2,+sse3,+sse4.1,+ssse3,+x87" "tune-cpu"="generic" }
bmanturner commented 2 years ago

I'm on an Intel Mac

guilhermeleobas commented 1 year ago

This bug happens as RBC is selecting system clang to compile C code. When the system clang version is greater than the one used by Numba/llvmlite, parts of the IR (i.e. new attributes) may not be recognized by the LLVM shipped with llvmlite.

This bug should be fixed in #520 and #521