Open MNLubov opened 1 year ago
chengchingwen
commented
1 year ago Hi Maxim,
I'm not sure why the error happened. It seems that LLVM does not know how to handle the pdep instruction while the cpu seems to support it. Does DoubleArrayTries.jl runs without error without PackageCompiler.jl? If it's a PackageCompiler-specific error, we should also open an issue there. I'll also see if there are any workaround I can do here.
MNLubov
commented
1 year ago Hi Peter,
Thank you for your response to my previous message. To provide further context, I have been experiencing issues while trying to build a binary image on AWS. I can confirm that on the local machine, the binary image is built without any issues. DoubleArrayTries.jl runs without error without PackageCompiler.jl.
chengchingwen
commented
1 year ago If I understand correctly, the error only happened on AWS with PackageCompiler.jl, but on local everything is fine? If so, could you check if the julia/LLVM version on AWS and your local machine are the same? Also, I would like to know if adding try block around those lines or forcing has_bmi2 return false avoiding the error. If these can be done, we can set some extra check or even a preference to turn on/off the pdep implementation.
MNLubov
commented
1 year ago After conducting experiments, I discovered that the error is quite unusual. I successfully built a binary image on AWS for a project that only has DoubleArrayTries, Transformers.jl, TextEncodeBase.jl as dependencies and nothing else. However, when attempting to build a binary image for the original project, it fails.
MNLubov
commented
1 year ago @chengchingwen
After conducting some experiments, I found that disabling the use of LLVM for pdep leads to a successful build of the binary image. Additionally, I discovered that a try-catch condition for calling pdep via LLVM results in the same error that I described in the issue. julia/LLVM version on AWS is 11.
chengchingwen
commented
1 year ago Can you share a minimal environment setting that could reproduce the error?
I found that disabling the use of LLVM for pdep leads to a successful build of the binary image
Did you do this by commenting out pdep implementation with llvmcall or you force the has_bmi2 return false? If it only works with commenting out pdep with LLVM, then it means we probably couldn't do anything on our side and we should report this error to PackageCompiler.jl
MNLubov
commented
1 year ago Thellvm error was connected to parameter cpu_target of the create_sysimage function:
PackageCompiler.create_sysimage(
project_name,
cpu_target="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)",
sysimage_path = joinpath(sysimage_dir, "image.so"),
precompile_execution_file = "test/runtests.jl",
)Removing cpu_target kwarg led to successful binary image build.
As for cpu_target value, it's standard value for the x86 architecture, see here
https://github.com/JuliaLang/PackageCompiler.jl/blob/df3543443b8b5740150e28a5638f7c635c2c61eb/src/PackageCompiler.jl#L33
Can you share a minimal environment setting that could reproduce the error?
project is build in Docker on AWS t3 instance using CentOS 7.9.2009 x86_64
Did you do this by commenting out pdep implementation with llvmcall or you force the has_bmi2 return false?
It worked either by setting has_bmi2 to false or by completely commenting out llvm implementation of pdep. Adding try-catch around llvm implementation of pdep led to the error.
So, summarizing all of the above, I suppose, it's PackageCompiler issue.
rssdev10
commented
1 year ago Hello, looks like we have some idea what is going on.
The concern is about LLVM and a minimal processor family provided by the argument cpu_target="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)". If we have a case of running LLVM with sandybridge target, definitely bmi2 is not supported. Right now we have a build agent VM with Skylake family and Intel(R) Xeon(R) Platinum 8259CL CPU. If we do not specify cpu_target, the PackageCompiler works without issues.
In the https://docs.julialang.org/en/v1/devdocs/sysimg/#Specifying-multiple-system-image-targets the only recommendation about cpu_target is present. Unfortunately even https://github.com/JuliaLang/julia/blob/0f5f62cf5a31b4a973ab0fae70988e35e2ea8da7/src/processor_x86.cpp#L196 doesn't answer how to specify haswell or broadwell as the minimal and the only required CPU family.
@KristofferC we need your help.
We cannot avoid use of cpu_target - we are using Docker images created inside an AWS VM with further deployment into an AWS cluster. Due to the use of virtual machines, we cannot guarantee a certain generation of physical CPUs. Sometimes, when we are building a Docker image, the build agent VM has a newer processor family than a Kubernetes cluster. After that, the error "unsupported processor type" appeared on deploy stage. When we have cpu_target hard restricted to the processor family, that is never happening.
rssdev10
commented
1 year ago https://github.com/chengchingwen/DoubleArrayTries.jl/blob/main/src/succinct.jl#L21
@static if has_bmi2()
select_in_word(x, k) = pdep_select_in_word(x, k)
pdep(x::UInt32, y::UInt32) = ccall("llvm.x86.bmi.pdep.32", llvmcall, UInt32, (UInt32, UInt32), x, y)
pdep(x::UInt64, y::UInt64) = ccall("llvm.x86.bmi.pdep.64", llvmcall, UInt64, (UInt64, UInt64), x, y)
else
select_in_word(x, k) = tabled_select_in_word(x, k)
pdep(x::T, y::T) where {T <: Union{UInt32, UInt64}} = _pdep(x, y)
end@MNLubov @rssdev10
Could you try using this branch and call DoubleArrayTries.set_native_pdep_support(false) to disable the pdep implementation with llvmcall and see if this work for you?
MNLubov
commented
1 year ago @chengchingwen
Disabling llvm implementation of the pdep with DoubleArrayTries.set_native_pdep_support(false) works. Thank you!
@chengchingwen Hi Peter,
I am encountering an issue with the DoubleArrayTries.jl package when building my Julia project. Specifically, I am getting an LLVM error related to the X86ISD::PDEP instruction in the
succinct.jlfile.Here are the specific error messages I am seeing:
Do you have any suggestions for how I can resolve this issue or any further troubleshooting steps I can try? I am happy to provide additional information or debug the issue further if needed.
Thank you for your help.
Sincerely, Maxim