Open suryajasper opened 1 year ago
@bjacob @MaheshRavishankar please let us know if you have any guidance
Thanks for the report. I've minimized the testcase and filed 2 issues as a result:
aarch64
case as in this original reprox86-64
case, running into a different error which aarch64
might eventually also run into once the immediate issue #14186 is resolved.The over-arching theme seems to be that this might be the first time we're compiling a FP16 Softmax, and some work is needed to make that work :-)
Also a note about the original testcase above:
The flag --iree-llvmcpu-target-cpu-features=host
is unwanted here. It tells iree-compile
to target the same CPU as it's running on. It only makes sense when the target triple matches the host architecture, which apparently it does not here: the target triple in the repro command line says aarch64
but the generated IR attached above has cpu_features
enumerating x86-64 features, apparently coming from that --iree-llvmcpu-target-cpu-features=host
flag, resulting in a log of warnings about unknown features for this architecture.
Note you have an easy way out here, which is to convert FP16->FP32 and back around each softmax. You can do that manually in the source IR or you could write a compiler pass for that if it doesn't exist already. @dcaballe might know.
- iree_linalg_ext.softmax on tensor<?xf16> causes linker error: undefined symbol:
fmaxf
#14187 , for thex86-64
case, running into a different error whichaarch64
might eventually also run into once the immediate issue
https://github.com/openxla/iree/pull/13808 should fix this issue.
@bjacob @MaheshRavishankar Please confirm if #13808 fixed this.
@bjacob @MaheshRavishankar Please confirm if #13808 fixed this.
It does not fixe the issue. It still trigger the issue mentioned in https://github.com/openxla/iree/issues/14186 We have to fix that one first.
Trying to compile FP16 quantized vicuna model on ARM64 CPU, but iree-compile is failing with LLVM error in one dispatch
Stack trace:
Failing dispatch MLIR (forward_dispatch_18)